[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:12:13 dist-test-slave-q276 krb5kdc[6993](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:12:13 dist-test-slave-q276 krb5kdc[6993](info): set up 2 sockets
Sep 24 21:12:13 dist-test-slave-q276 krb5kdc[6993](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260924 21:12:14.200217 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:12:14 dist-test-slave-q276 krb5kdc[6993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:12:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:12:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.210.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
I20260924 21:12:14.317746 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:44493
--webserver_interface=127.6.210.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:39103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:44493
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:14.397418 7006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:14.397657 7006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:14.397696 7006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:14.398936 7006 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:12:14.398998 7006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:14.399022 7006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:14.399045 7006 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:12:14.399091 7006 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:12:14.400593 7006 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:39103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:44493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:44493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:14.400866 7006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:14.401095 7006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:14.406328 7012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:14.406347 7006 server_base.cc:1034] running on GCE node
W20260924 21:12:14.406404 7014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:14.406466 7011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:14.406810 7006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:14.407097 7006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:14.408264 7006 hybrid_clock.cc:648] HybridClock initialized: now 1790284334408241 us; error 59 us; skew 500 ppm
Sep 24 21:12:14 dist-test-slave-q276 krb5kdc[6993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284334, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:14.410827 7006 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:12:14.411343 7006 webserver.cc:460] Webserver started at http://127.6.210.190:46499/ using document root <none> and password file <none>
I20260924 21:12:14.411593 7006 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:14.411659 7006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:14.411810 7006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:14.412868 7006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "034c0277de4c4d71bc50f01c152bfd81"
format_stamp: "Formatted at 2026-09-24 21:12:14 on dist-test-slave-q276"
server_key: "e74fe6014c84e094c37bb141f25830bb"
server_key_iv: "7999b6c1d7eed7790030e82b1ab94651"
server_key_version: "encryptionkey@0"
I20260924 21:12:14.413257 7006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "034c0277de4c4d71bc50f01c152bfd81"
format_stamp: "Formatted at 2026-09-24 21:12:14 on dist-test-slave-q276"
server_key: "e74fe6014c84e094c37bb141f25830bb"
server_key_iv: "7999b6c1d7eed7790030e82b1ab94651"
server_key_version: "encryptionkey@0"
I20260924 21:12:14.414597 7006 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:12:14.415411 7021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:14.415612 7006 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:12:14.415717 7006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "034c0277de4c4d71bc50f01c152bfd81"
format_stamp: "Formatted at 2026-09-24 21:12:14 on dist-test-slave-q276"
server_key: "e74fe6014c84e094c37bb141f25830bb"
server_key_iv: "7999b6c1d7eed7790030e82b1ab94651"
server_key_version: "encryptionkey@0"
I20260924 21:12:14.415788 7006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:14.428391 7006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:14.430323 7006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:14.430481 7006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:14.434427 7006 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:44493
I20260924 21:12:14.434504 7073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:44493 every 8 connection(s)
I20260924 21:12:14.434854 7006 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:12:14.435676 7074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:14.437912 7074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 034c0277de4c4d71bc50f01c152bfd81: Bootstrap starting.
I20260924 21:12:14.438575 7074 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 034c0277de4c4d71bc50f01c152bfd81: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:14.438870 7074 log.cc:824] T 00000000000000000000000000000000 P 034c0277de4c4d71bc50f01c152bfd81: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:14.439559 7074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 034c0277de4c4d71bc50f01c152bfd81: No bootstrap required, opened a new log
I20260924 21:12:14.441164 7074 raft_consensus.cc:348] T 00000000000000000000000000000000 P 034c0277de4c4d71bc50f01c152bfd81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034c0277de4c4d71bc50f01c152bfd81" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 44493 } }
I20260924 21:12:14.441304 7074 raft_consensus.cc:374] T 00000000000000000000000000000000 P 034c0277de4c4d71bc50f01c152bfd81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:14.441327 7074 raft_consensus.cc:729] T 00000000000000000000000000000000 P 034c0277de4c4d71bc50f01c152bfd81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 034c0277de4c4d71bc50f01c152bfd81, State: Initialized, Role: FOLLOWER
I20260924 21:12:14.441475 7074 consensus_queue.cc:260] T 00000000000000000000000000000000 P 034c0277de4c4d71bc50f01c152bfd81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034c0277de4c4d71bc50f01c152bfd81" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 44493 } }
I20260924 21:12:14.441558 7074 raft_consensus.cc:388] T 00000000000000000000000000000000 P 034c0277de4c4d71bc50f01c152bfd81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:12:14.441581 7074 raft_consensus.cc:482] T 00000000000000000000000000000000 P 034c0277de4c4d71bc50f01c152bfd81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:12:14.441654 7074 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 034c0277de4c4d71bc50f01c152bfd81 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:14.442225 7074 raft_consensus.cc:504] T 00000000000000000000000000000000 P 034c0277de4c4d71bc50f01c152bfd81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034c0277de4c4d71bc50f01c152bfd81" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 44493 } }
I20260924 21:12:14.442255 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 7006
I20260924 21:12:14.442412 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:12:14.442418 7074 leader_election.cc:302] T 00000000000000000000000000000000 P 034c0277de4c4d71bc50f01c152bfd81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 034c0277de4c4d71bc50f01c152bfd81; no voters:
I20260924 21:12:14.442631 7074 leader_election.cc:288] T 00000000000000000000000000000000 P 034c0277de4c4d71bc50f01c152bfd81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:12:14.442648 7077 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 034c0277de4c4d71bc50f01c152bfd81 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:14.442898 7074 sys_catalog.cc:564] T 00000000000000000000000000000000 P 034c0277de4c4d71bc50f01c152bfd81 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:12:14.443174 6986 external_mini_cluster.cc:1428] Setting key cd65cc2b66aecabee9519b6bd8721a91
I20260924 21:12:14.444416 7077 raft_consensus.cc:686] T 00000000000000000000000000000000 P 034c0277de4c4d71bc50f01c152bfd81 [term 1 LEADER]: Becoming Leader. State: Replica: 034c0277de4c4d71bc50f01c152bfd81, State: Running, Role: LEADER
I20260924 21:12:14.444574 7077 consensus_queue.cc:237] T 00000000000000000000000000000000 P 034c0277de4c4d71bc50f01c152bfd81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034c0277de4c4d71bc50f01c152bfd81" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 44493 } }
I20260924 21:12:14.445122 7091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 034c0277de4c4d71bc50f01c152bfd81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 034c0277de4c4d71bc50f01c152bfd81. Latest consensus state: current_term: 1 leader_uuid: "034c0277de4c4d71bc50f01c152bfd81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034c0277de4c4d71bc50f01c152bfd81" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 44493 } } }
I20260924 21:12:14.445154 7078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 034c0277de4c4d71bc50f01c152bfd81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "034c0277de4c4d71bc50f01c152bfd81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034c0277de4c4d71bc50f01c152bfd81" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 44493 } } }
I20260924 21:12:14.445381 7078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 034c0277de4c4d71bc50f01c152bfd81 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:14.445598 7091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 034c0277de4c4d71bc50f01c152bfd81 [sys.catalog]: This master's current role is: LEADER
W20260924 21:12:14.445971 7093 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 034c0277de4c4d71bc50f01c152bfd81: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 21:12:14.446035 7093 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 21:12:14.446116 7094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:12:14.446316 7094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:12:14.447741 7094 catalog_manager.cc:1347] Generated new cluster ID: 0674b313f1e2478eb078e91f9b0b9b02
I20260924 21:12:14.447795 7094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 21:12:14 dist-test-slave-q276 krb5kdc[6993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:14.456627 7094 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:12:14.457305 7094 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:12:14.458804 7088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:14.444055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45244 (local address 127.6.210.190:44493)
0924 21:12:14.444272 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:14.444281 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:14.444588 (+ 307us) server_negotiation.cc:407] Connection header received
0924 21:12:14.447119 (+ 2531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:14.447125 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:14.447203 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:14.447354 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:14.448951 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:14.449903 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:14.451042 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:14.451474 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:14.451595 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:14.454948 (+ 3353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:14.454951 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:14.454952 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:14.454961 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:14.456874 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:14.457303 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:14.457306 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:14.457306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:14.457464 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:14.457717 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:14.457719 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:14.457719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:14.457943 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:14.458129 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:14.458426 (+ 297us) server_negotiation.cc:299] Negotiation successful
0924 21:12:14.458565 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":101,"threads_started":1}
I20260924 21:12:14.462886 7094 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 034c0277de4c4d71bc50f01c152bfd81: Generated new TSK 0
I20260924 21:12:14.463064 7094 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.210.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
I20260924 21:12:14.493547 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.129:0
--local_ip_for_outbound_sockets=127.6.210.129
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:44493
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:39103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:14.591652 7100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:14.591858 7100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:14.591882 7100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:14.593051 7100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:14.593091 7100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:14.593117 7100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.129
I20260924 21:12:14.594633 7100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:39103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:44493
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.129
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:14.594875 7100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:14.595157 7100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:14.601034 7106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:14.601171 7108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:14.601337 7105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:14.601231 7100 server_base.cc:1034] running on GCE node
I20260924 21:12:14.601646 7100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:14.601917 7100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:14.603121 7100 hybrid_clock.cc:648] HybridClock initialized: now 1790284334603101 us; error 33 us; skew 500 ppm
Sep 24 21:12:14 dist-test-slave-q276 krb5kdc[6993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284334, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:14.606158 7100 init.cc:377] Logged in from keytab as kudu/127.6.210.129@KRBTEST.COM (short username kudu)
I20260924 21:12:14.606660 7100 webserver.cc:460] Webserver started at http://127.6.210.129:38099/ using document root <none> and password file <none>
I20260924 21:12:14.606916 7100 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:14.606990 7100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:14.607133 7100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:14.608194 7100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-0/data/instance:
uuid: "a2cd907e92ba4424a94a2d4478ebf3dc"
format_stamp: "Formatted at 2026-09-24 21:12:14 on dist-test-slave-q276"
server_key: "1ad7b6cd56f6f031b9be1896d746712c"
server_key_iv: "b5df95acb7a76ed55368fbb0311030a3"
server_key_version: "encryptionkey@0"
I20260924 21:12:14.608579 7100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance:
uuid: "a2cd907e92ba4424a94a2d4478ebf3dc"
format_stamp: "Formatted at 2026-09-24 21:12:14 on dist-test-slave-q276"
server_key: "1ad7b6cd56f6f031b9be1896d746712c"
server_key_iv: "b5df95acb7a76ed55368fbb0311030a3"
server_key_version: "encryptionkey@0"
I20260924 21:12:14.609982 7100 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:12:14.610829 7115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:14.611033 7100 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 21:12:14.611131 7100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-0/wal
uuid: "a2cd907e92ba4424a94a2d4478ebf3dc"
format_stamp: "Formatted at 2026-09-24 21:12:14 on dist-test-slave-q276"
server_key: "1ad7b6cd56f6f031b9be1896d746712c"
server_key_iv: "b5df95acb7a76ed55368fbb0311030a3"
server_key_version: "encryptionkey@0"
I20260924 21:12:14.611204 7100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:14.624866 7100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:14.626842 7100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:14.626976 7100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:14.627172 7100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:14.627516 7100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:14.627553 7100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:14.627601 7100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:14.627632 7100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:14.633790 7100 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.129:34359
I20260924 21:12:14.633863 7228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.129:34359 every 8 connection(s)
I20260924 21:12:14.634210 7100 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
I20260924 21:12:14.639091 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 7100
I20260924 21:12:14.639230 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance
I20260924 21:12:14.639392 6986 external_mini_cluster.cc:1428] Setting key 30fd9ce77cdcda1b939432bcfd6c5b06
Sep 24 21:12:14 dist-test-slave-q276 krb5kdc[6993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284334, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:14.644134 7088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:14.634994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:56739 (local address 127.6.210.190:44493)
0924 21:12:14.635050 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:14.635053 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:14.635294 (+ 241us) server_negotiation.cc:407] Connection header received
0924 21:12:14.635803 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:14.635804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:14.635815 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:14.635875 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:14.636863 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:14.637363 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:14.638366 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:14.638571 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:14.638625 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:14.640893 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:14.640894 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:14.640895 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:14.640902 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:14.642459 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:14.643011 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:14.643013 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:14.643013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:14.643075 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:14.643325 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:14.643326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:14.643326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:14.643446 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:14.643554 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:14.643880 (+ 326us) server_negotiation.cc:299] Negotiation successful
0924 21:12:14.644045 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 21:12:14.644236 7231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:14.634983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:44493 (local address 127.6.210.129:56739)
0924 21:12:14.635160 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:14.635196 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:14.635666 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:14.635955 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:14.635958 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:14.636572 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:14.636772 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:14.636778 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:14.637445 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:14.637446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:14.638238 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:14.638240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:14.638641 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:14.638642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:14.638747 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:14.639267 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:14.639278 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:14.640807 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:14.642540 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:14.642546 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:14.642548 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:14.642855 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:14.643142 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:14.643143 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:14.643144 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:14.643257 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:14.643556 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:14.643563 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:14.643721 (+ 158us) client_negotiation.cc:769] Sending connection context
0924 21:12:14.643848 (+ 127us) client_negotiation.cc:240] Negotiation successful
0924 21:12:14.644010 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":94,"threads_started":1}
I20260924 21:12:14.645009 7229 heartbeater.cc:347] Connected to a master server at 127.6.210.190:44493
I20260924 21:12:14.645128 7229 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:14.645344 7229 heartbeater.cc:511] Master 127.6.210.190:44493 requested a full tablet report, sending...
I20260924 21:12:14.645860 7038 ts_manager.cc:194] Registered new tserver with Master: a2cd907e92ba4424a94a2d4478ebf3dc (127.6.210.129:34359)
I20260924 21:12:14.646580 7038 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.129@KRBTEST.COM'} at 127.6.210.129:56739
WARNING: no policy specified for kudu/127.6.210.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
I20260924 21:12:14.665340 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.130:0
--local_ip_for_outbound_sockets=127.6.210.130
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:44493
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:39103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:14.746869 7234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:14.747085 7234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:14.747108 7234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:14.748188 7234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:14.748224 7234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:14.748250 7234 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.130
I20260924 21:12:14.749680 7234 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:39103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:44493
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.130
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:14.749859 7234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:14.750041 7234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:14.754913 7242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:14.754884 7240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:14.754866 7239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:14.755085 7234 server_base.cc:1034] running on GCE node
I20260924 21:12:14.755264 7234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:14.755520 7234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:14.756662 7234 hybrid_clock.cc:648] HybridClock initialized: now 1790284334756646 us; error 33 us; skew 500 ppm
Sep 24 21:12:14 dist-test-slave-q276 krb5kdc[6993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284334, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:14.759166 7234 init.cc:377] Logged in from keytab as kudu/127.6.210.130@KRBTEST.COM (short username kudu)
I20260924 21:12:14.759680 7234 webserver.cc:460] Webserver started at http://127.6.210.130:36997/ using document root <none> and password file <none>
I20260924 21:12:14.759900 7234 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:14.759964 7234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:14.760113 7234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:14.761162 7234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-1/data/instance:
uuid: "3d87d8de6df6468ebb8746a8b9ab0e9b"
format_stamp: "Formatted at 2026-09-24 21:12:14 on dist-test-slave-q276"
server_key: "851488d34f58556d5a310f9db714e56a"
server_key_iv: "2f0c299ca17e8855246c0601bb6aa2d1"
server_key_version: "encryptionkey@0"
I20260924 21:12:14.761538 7234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance:
uuid: "3d87d8de6df6468ebb8746a8b9ab0e9b"
format_stamp: "Formatted at 2026-09-24 21:12:14 on dist-test-slave-q276"
server_key: "851488d34f58556d5a310f9db714e56a"
server_key_iv: "2f0c299ca17e8855246c0601bb6aa2d1"
server_key_version: "encryptionkey@0"
I20260924 21:12:14.762914 7234 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260924 21:12:14.763589 7249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:14.763777 7234 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:12:14.763877 7234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-1/wal
uuid: "3d87d8de6df6468ebb8746a8b9ab0e9b"
format_stamp: "Formatted at 2026-09-24 21:12:14 on dist-test-slave-q276"
server_key: "851488d34f58556d5a310f9db714e56a"
server_key_iv: "2f0c299ca17e8855246c0601bb6aa2d1"
server_key_version: "encryptionkey@0"
I20260924 21:12:14.763949 7234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:14.775314 7234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:14.777657 7234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:14.777824 7234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:14.778497 7234 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:14.778810 7234 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:14.778863 7234 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:14.778908 7234 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:14.778954 7234 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:14.785509 7234 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.130:33409
I20260924 21:12:14.785549 7362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.130:33409 every 8 connection(s)
I20260924 21:12:14.785856 7234 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
I20260924 21:12:14.789979 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 7234
I20260924 21:12:14.790086 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance
I20260924 21:12:14.790246 6986 external_mini_cluster.cc:1428] Setting key af3ea2f965727f47701b25b79d3ecf40
Sep 24 21:12:14 dist-test-slave-q276 krb5kdc[6993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284334, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:14.795920 7088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:14.786600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:57433 (local address 127.6.210.190:44493)
0924 21:12:14.786687 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:14.786690 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:14.786872 (+ 182us) server_negotiation.cc:407] Connection header received
0924 21:12:14.787393 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:14.787395 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:14.787407 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:14.787459 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:14.788475 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:14.788972 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:14.789966 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:14.790240 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:14.790361 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:14.792816 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:14.792817 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:14.792818 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:14.792825 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:14.794391 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:14.794914 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:14.794915 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:14.794915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:14.794967 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:14.795259 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:14.795259 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:14.795260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:14.795362 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:14.795412 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:14.795780 (+ 368us) server_negotiation.cc:299] Negotiation successful
0924 21:12:14.795841 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 21:12:14.796267 7365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:14.786554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:44493 (local address 127.6.210.130:57433)
0924 21:12:14.786740 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:14.786775 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:14.787277 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:14.787547 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:14.787551 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:14.788172 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:14.788365 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:14.788369 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:14.789063 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:14.789065 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:14.789859 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:14.789861 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:14.790420 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:14.790421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:14.790536 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:14.791140 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:14.791149 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:14.792723 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:14.794471 (+ 1748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:14.794478 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:14.794480 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:14.794808 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:14.795033 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:14.795034 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:14.795034 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:14.795188 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:14.795435 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:14.795441 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:14.795626 (+ 185us) client_negotiation.cc:769] Sending connection context
0924 21:12:14.795756 (+ 130us) client_negotiation.cc:240] Negotiation successful
0924 21:12:14.795979 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":81,"threads_started":1}
I20260924 21:12:14.797235 7363 heartbeater.cc:347] Connected to a master server at 127.6.210.190:44493
I20260924 21:12:14.797343 7363 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:14.797554 7363 heartbeater.cc:511] Master 127.6.210.190:44493 requested a full tablet report, sending...
I20260924 21:12:14.797973 7038 ts_manager.cc:194] Registered new tserver with Master: 3d87d8de6df6468ebb8746a8b9ab0e9b (127.6.210.130:33409)
I20260924 21:12:14.798408 7038 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.130@KRBTEST.COM'} at 127.6.210.130:57433
WARNING: no policy specified for kudu/127.6.210.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
I20260924 21:12:14.816938 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.131:0
--local_ip_for_outbound_sockets=127.6.210.131
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:44493
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:39103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:14.899361 7368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:14.899675 7368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:14.899703 7368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:14.901118 7368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:14.901170 7368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:14.901199 7368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.131
I20260924 21:12:14.902827 7368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:39103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:44493
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.131
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:14.903080 7368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:14.903347 7368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:14.908797 7374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:14.908816 7373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:14.908846 7368 server_base.cc:1034] running on GCE node
W20260924 21:12:14.908813 7376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:14.909293 7368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:14.909554 7368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:14.910722 7368 hybrid_clock.cc:648] HybridClock initialized: now 1790284334910709 us; error 25 us; skew 500 ppm
Sep 24 21:12:14 dist-test-slave-q276 krb5kdc[6993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284334, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:14.913707 7368 init.cc:377] Logged in from keytab as kudu/127.6.210.131@KRBTEST.COM (short username kudu)
I20260924 21:12:14.914175 7368 webserver.cc:460] Webserver started at http://127.6.210.131:44033/ using document root <none> and password file <none>
I20260924 21:12:14.914469 7368 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:14.914590 7368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:14.914734 7368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:14.915907 7368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-2/data/instance:
uuid: "40248e53f9194bbaae4cca5aff21139e"
format_stamp: "Formatted at 2026-09-24 21:12:14 on dist-test-slave-q276"
server_key: "bcfc05f7bf6bbfb55dbc48512f457595"
server_key_iv: "6b0430bec4ea1afd644dff9202ef5cd1"
server_key_version: "encryptionkey@0"
I20260924 21:12:14.916268 7368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance:
uuid: "40248e53f9194bbaae4cca5aff21139e"
format_stamp: "Formatted at 2026-09-24 21:12:14 on dist-test-slave-q276"
server_key: "bcfc05f7bf6bbfb55dbc48512f457595"
server_key_iv: "6b0430bec4ea1afd644dff9202ef5cd1"
server_key_version: "encryptionkey@0"
I20260924 21:12:14.917608 7368 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:12:14.918443 7383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:14.918689 7368 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:14.918762 7368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-2/wal
uuid: "40248e53f9194bbaae4cca5aff21139e"
format_stamp: "Formatted at 2026-09-24 21:12:14 on dist-test-slave-q276"
server_key: "bcfc05f7bf6bbfb55dbc48512f457595"
server_key_iv: "6b0430bec4ea1afd644dff9202ef5cd1"
server_key_version: "encryptionkey@0"
I20260924 21:12:14.918849 7368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:14.927618 7368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:14.929514 7368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:14.929669 7368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:14.929905 7368 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:14.930212 7368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:14.930246 7368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:14.930351 7368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:14.930372 7368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:14.936558 7368 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.131:44919
I20260924 21:12:14.936616 7496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.131:44919 every 8 connection(s)
I20260924 21:12:14.936947 7368 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
I20260924 21:12:14.941687 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 7368
I20260924 21:12:14.941797 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance
I20260924 21:12:14.941972 6986 external_mini_cluster.cc:1428] Setting key 96d62fdd9541959f7796627b056f5fbf
Sep 24 21:12:14 dist-test-slave-q276 krb5kdc[6993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284334, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:14.946509 7088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:14.937673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:39265 (local address 127.6.210.190:44493)
0924 21:12:14.937739 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:14.937742 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:14.937960 (+ 218us) server_negotiation.cc:407] Connection header received
0924 21:12:14.938516 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:14.938517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:14.938527 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:14.938581 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:14.939601 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:14.940088 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:14.941022 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:14.941200 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:14.941254 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:14.943910 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:14.943912 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:14.943912 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:14.943918 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:14.945150 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:14.945611 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:14.945612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:14.945612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:14.945700 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:14.945943 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:14.945944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:14.945944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:14.946045 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:14.946093 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:14.946381 (+ 288us) server_negotiation.cc:299] Negotiation successful
0924 21:12:14.946449 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 21:12:14.946748 7499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:14.937658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:44493 (local address 127.6.210.131:39265)
0924 21:12:14.937841 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:14.937877 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:14.938431 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:14.938653 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:14.938656 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:14.939229 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:14.939515 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:14.939521 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:14.940164 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:14.940164 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:14.940947 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:14.940949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:14.941272 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:14.941273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:14.941367 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:14.941978 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:14.941986 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:14.943824 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:14.945223 (+ 1399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:14.945247 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:14.945249 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:14.945531 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:14.945776 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:14.945777 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:14.945778 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:14.945885 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:14.946098 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:14.946101 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:14.946210 (+ 109us) client_negotiation.cc:769] Sending connection context
0924 21:12:14.946362 (+ 152us) client_negotiation.cc:240] Negotiation successful
0924 21:12:14.946492 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":99,"threads_started":1}
I20260924 21:12:14.947204 7497 heartbeater.cc:347] Connected to a master server at 127.6.210.190:44493
I20260924 21:12:14.947309 7497 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:14.947501 7497 heartbeater.cc:511] Master 127.6.210.190:44493 requested a full tablet report, sending...
I20260924 21:12:14.947847 7038 ts_manager.cc:194] Registered new tserver with Master: 40248e53f9194bbaae4cca5aff21139e (127.6.210.131:44919)
I20260924 21:12:14.948217 7038 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.131@KRBTEST.COM'} at 127.6.210.131:39265
I20260924 21:12:14.954113 6986 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 24 21:12:14 dist-test-slave-q276 krb5kdc[6993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284334, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 21:12:14 dist-test-slave-q276 krb5kdc[6993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284334, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.210.129@KRBTEST.COM
I20260924 21:12:14.976212 7504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:14.969036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34928 (local address 127.6.210.129:34359)
0924 21:12:14.969197 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:14.969202 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:14.969220 (+ 18us) server_negotiation.cc:407] Connection header received
0924 21:12:14.969246 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:14.969247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:14.969261 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:14.969329 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:14.969813 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:14.970646 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:14.971338 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:14.971655 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:14.971708 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:14.973795 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:14.973802 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:14.973802 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:14.973808 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:14.975254 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:14.975553 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:14.975556 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:14.975556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:14.975608 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:14.975769 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:14.975769 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:14.975769 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:14.975890 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:14.975988 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:14.976076 (+ 88us) server_negotiation.cc:299] Negotiation successful
0924 21:12:14.976134 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":99,"threads_started":1}
W20260924 21:12:14.976611 7143 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34928
Sep 24 21:12:14 dist-test-slave-q276 krb5kdc[6993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 21:12:14 dist-test-slave-q276 krb5kdc[6993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.129@KRBTEST.COM
I20260924 21:12:14.997864 7504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:14.991033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34936 (local address 127.6.210.129:34359)
0924 21:12:14.991118 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:14.991120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:14.991190 (+ 70us) server_negotiation.cc:407] Connection header received
0924 21:12:14.991239 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:14.991239 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:14.991250 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:14.991300 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:14.991797 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:14.992279 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:14.992973 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:14.993162 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:14.993213 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:14.995455 (+ 2242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:14.995456 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:14.995456 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:14.995463 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:14.996675 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:14.997031 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:14.997033 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:14.997033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:14.997097 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:14.997314 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:14.997315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:14.997316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:14.997461 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:14.997567 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:14.997704 (+ 137us) server_negotiation.cc:299] Negotiation successful
0924 21:12:14.997781 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260924 21:12:14.998474 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 7100
I20260924 21:12:15.004796 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 7234
I20260924 21:12:15.009454 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 7368
I20260924 21:12:15.013661 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 7006
2026-09-24T21:12:15Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1859 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:12:15 dist-test-slave-q276 krb5kdc[7511](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:12:15 dist-test-slave-q276 krb5kdc[7511](info): set up 2 sockets
Sep 24 21:12:15 dist-test-slave-q276 krb5kdc[7511](info): commencing operation
krb5kdc: starting...
W20260924 21:12:16.126935 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:12:16 dist-test-slave-q276 krb5kdc[7511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:12:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:12:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.210.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
I20260924 21:12:16.246167 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:41659
--webserver_interface=127.6.210.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:45581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:41659
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:16.328279 7524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:16.328487 7524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:16.328510 7524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:16.329677 7524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:12:16.329720 7524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:16.329733 7524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:16.329746 7524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:12:16.329758 7524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:12:16.331276 7524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:45581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:41659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:41659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:16.331506 7524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:16.331760 7524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:16.336639 7532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:16.336647 7530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:16.336651 7529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:16.336728 7524 server_base.cc:1034] running on GCE node
I20260924 21:12:16.337105 7524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:16.337394 7524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:16.338572 7524 hybrid_clock.cc:648] HybridClock initialized: now 1790284336338550 us; error 36 us; skew 500 ppm
Sep 24 21:12:16 dist-test-slave-q276 krb5kdc[7511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284336, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:16.341004 7524 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:12:16.341424 7524 webserver.cc:460] Webserver started at http://127.6.210.190:33641/ using document root <none> and password file <none>
I20260924 21:12:16.341660 7524 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:16.341727 7524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:16.341876 7524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:16.343214 7524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "e81e3130eb7f40e88a747f24e038475a"
format_stamp: "Formatted at 2026-09-24 21:12:16 on dist-test-slave-q276"
server_key: "ce356ab4ef69231b56093455c059318f"
server_key_iv: "f2c6e943d21ee2339aee04e713a4db43"
server_key_version: "encryptionkey@0"
I20260924 21:12:16.343782 7524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "e81e3130eb7f40e88a747f24e038475a"
format_stamp: "Formatted at 2026-09-24 21:12:16 on dist-test-slave-q276"
server_key: "ce356ab4ef69231b56093455c059318f"
server_key_iv: "f2c6e943d21ee2339aee04e713a4db43"
server_key_version: "encryptionkey@0"
I20260924 21:12:16.345184 7524 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:12:16.345956 7539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:16.346158 7524 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:16.346293 7524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "e81e3130eb7f40e88a747f24e038475a"
format_stamp: "Formatted at 2026-09-24 21:12:16 on dist-test-slave-q276"
server_key: "ce356ab4ef69231b56093455c059318f"
server_key_iv: "f2c6e943d21ee2339aee04e713a4db43"
server_key_version: "encryptionkey@0"
I20260924 21:12:16.346377 7524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:16.360040 7524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:16.362044 7524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:16.362154 7524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:16.365701 7524 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:41659
I20260924 21:12:16.365756 7591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:41659 every 8 connection(s)
I20260924 21:12:16.366082 7524 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:12:16.366971 7592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:16.369220 7592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e81e3130eb7f40e88a747f24e038475a: Bootstrap starting.
I20260924 21:12:16.369794 7592 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e81e3130eb7f40e88a747f24e038475a: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:16.370078 7592 log.cc:824] T 00000000000000000000000000000000 P e81e3130eb7f40e88a747f24e038475a: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:16.370824 7592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e81e3130eb7f40e88a747f24e038475a: No bootstrap required, opened a new log
I20260924 21:12:16.370988 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 7524
I20260924 21:12:16.371068 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:12:16.371217 6986 external_mini_cluster.cc:1428] Setting key e41f409ec54309317c231e7fea731ba5
I20260924 21:12:16.372732 7592 raft_consensus.cc:348] T 00000000000000000000000000000000 P e81e3130eb7f40e88a747f24e038475a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e81e3130eb7f40e88a747f24e038475a" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 41659 } }
I20260924 21:12:16.372889 7592 raft_consensus.cc:374] T 00000000000000000000000000000000 P e81e3130eb7f40e88a747f24e038475a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:16.372933 7592 raft_consensus.cc:729] T 00000000000000000000000000000000 P e81e3130eb7f40e88a747f24e038475a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e81e3130eb7f40e88a747f24e038475a, State: Initialized, Role: FOLLOWER
I20260924 21:12:16.373054 7592 consensus_queue.cc:260] T 00000000000000000000000000000000 P e81e3130eb7f40e88a747f24e038475a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e81e3130eb7f40e88a747f24e038475a" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 41659 } }
I20260924 21:12:16.373148 7592 raft_consensus.cc:388] T 00000000000000000000000000000000 P e81e3130eb7f40e88a747f24e038475a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:12:16.373193 7592 raft_consensus.cc:482] T 00000000000000000000000000000000 P e81e3130eb7f40e88a747f24e038475a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:12:16.373262 7592 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e81e3130eb7f40e88a747f24e038475a [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:16.374128 7592 raft_consensus.cc:504] T 00000000000000000000000000000000 P e81e3130eb7f40e88a747f24e038475a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e81e3130eb7f40e88a747f24e038475a" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 41659 } }
I20260924 21:12:16.374243 7592 leader_election.cc:302] T 00000000000000000000000000000000 P e81e3130eb7f40e88a747f24e038475a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e81e3130eb7f40e88a747f24e038475a; no voters:
I20260924 21:12:16.374455 7592 leader_election.cc:288] T 00000000000000000000000000000000 P e81e3130eb7f40e88a747f24e038475a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:12:16.374473 7597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e81e3130eb7f40e88a747f24e038475a [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:16.374588 7597 raft_consensus.cc:686] T 00000000000000000000000000000000 P e81e3130eb7f40e88a747f24e038475a [term 1 LEADER]: Becoming Leader. State: Replica: e81e3130eb7f40e88a747f24e038475a, State: Running, Role: LEADER
I20260924 21:12:16.374668 7597 consensus_queue.cc:237] T 00000000000000000000000000000000 P e81e3130eb7f40e88a747f24e038475a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e81e3130eb7f40e88a747f24e038475a" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 41659 } }
I20260924 21:12:16.374715 7592 sys_catalog.cc:564] T 00000000000000000000000000000000 P e81e3130eb7f40e88a747f24e038475a [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:12:16.375111 7599 sys_catalog.cc:455] T 00000000000000000000000000000000 P e81e3130eb7f40e88a747f24e038475a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e81e3130eb7f40e88a747f24e038475a. Latest consensus state: current_term: 1 leader_uuid: "e81e3130eb7f40e88a747f24e038475a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e81e3130eb7f40e88a747f24e038475a" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 41659 } } }
I20260924 21:12:16.375128 7598 sys_catalog.cc:455] T 00000000000000000000000000000000 P e81e3130eb7f40e88a747f24e038475a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e81e3130eb7f40e88a747f24e038475a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e81e3130eb7f40e88a747f24e038475a" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 41659 } } }
I20260924 21:12:16.375283 7599 sys_catalog.cc:458] T 00000000000000000000000000000000 P e81e3130eb7f40e88a747f24e038475a [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:16.375306 7598 sys_catalog.cc:458] T 00000000000000000000000000000000 P e81e3130eb7f40e88a747f24e038475a [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:16.375631 7606 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:12:16.376140 7606 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 24 21:12:16 dist-test-slave-q276 krb5kdc[7511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:16.379050 7606 catalog_manager.cc:1347] Generated new cluster ID: c1fefbfb9a124c5c95d50149dbdcb72a
I20260924 21:12:16.379103 7606 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:12:16.382495 7596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:16.371742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51722 (local address 127.6.210.190:41659)
0924 21:12:16.371924 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:16.371934 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:16.371967 (+ 33us) server_negotiation.cc:407] Connection header received
0924 21:12:16.372713 (+ 746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:16.372719 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:16.372765 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:16.372903 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:16.373603 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.374692 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.375743 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.376364 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.376827 (+ 463us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:16.379079 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:16.379085 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:16.379085 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:16.379120 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:16.380827 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:16.381229 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:16.381232 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:16.381232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:16.381340 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:16.381562 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:16.381563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:16.381563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:16.381792 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:16.381967 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:16.382061 (+ 94us) server_negotiation.cc:299] Negotiation successful
0924 21:12:16.382335 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":73,"threads_started":1}
I20260924 21:12:16.397187 7606 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:12:16.397889 7606 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:12:16.407492 7606 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e81e3130eb7f40e88a747f24e038475a: Generated new TSK 0
I20260924 21:12:16.407723 7606 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.210.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
I20260924 21:12:16.441485 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.129:0
--local_ip_for_outbound_sockets=127.6.210.129
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:41659
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:45581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:16.523687 7618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:16.523892 7618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:16.523921 7618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:16.525446 7618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:16.525513 7618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:16.525547 7618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.129
I20260924 21:12:16.527350 7618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:45581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:41659
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.129
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:16.527647 7618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:16.527930 7618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:16.533470 7624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:16.533455 7618 server_base.cc:1034] running on GCE node
W20260924 21:12:16.533423 7626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:16.533398 7623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:16.533954 7618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:16.534166 7618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:16.535303 7618 hybrid_clock.cc:648] HybridClock initialized: now 1790284336535304 us; error 42 us; skew 500 ppm
Sep 24 21:12:16 dist-test-slave-q276 krb5kdc[7511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284336, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:16.538201 7618 init.cc:377] Logged in from keytab as kudu/127.6.210.129@KRBTEST.COM (short username kudu)
I20260924 21:12:16.538671 7618 webserver.cc:460] Webserver started at http://127.6.210.129:46159/ using document root <none> and password file <none>
I20260924 21:12:16.538925 7618 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:16.538996 7618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:16.539147 7618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:16.540201 7618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-0/data/instance:
uuid: "f1d23c1b9b67452199e6e2c80e899bf7"
format_stamp: "Formatted at 2026-09-24 21:12:16 on dist-test-slave-q276"
server_key: "c46a2c1776f13b2c3c7ea8616b9a2901"
server_key_iv: "3d15de1ec2f9c1b630c18128433a45bd"
server_key_version: "encryptionkey@0"
I20260924 21:12:16.540563 7618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance:
uuid: "f1d23c1b9b67452199e6e2c80e899bf7"
format_stamp: "Formatted at 2026-09-24 21:12:16 on dist-test-slave-q276"
server_key: "c46a2c1776f13b2c3c7ea8616b9a2901"
server_key_iv: "3d15de1ec2f9c1b630c18128433a45bd"
server_key_version: "encryptionkey@0"
I20260924 21:12:16.541900 7618 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:12:16.542675 7633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:16.542848 7618 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:16.542946 7618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-0/wal
uuid: "f1d23c1b9b67452199e6e2c80e899bf7"
format_stamp: "Formatted at 2026-09-24 21:12:16 on dist-test-slave-q276"
server_key: "c46a2c1776f13b2c3c7ea8616b9a2901"
server_key_iv: "3d15de1ec2f9c1b630c18128433a45bd"
server_key_version: "encryptionkey@0"
I20260924 21:12:16.543026 7618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:16.559062 7618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:16.560910 7618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:16.561077 7618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:16.561321 7618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:16.561638 7618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:16.561694 7618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:16.561734 7618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:16.561762 7618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:16.567684 7618 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.129:40859
I20260924 21:12:16.567732 7746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.129:40859 every 8 connection(s)
I20260924 21:12:16.568048 7618 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
Sep 24 21:12:16 dist-test-slave-q276 krb5kdc[7511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284336, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:16.576858 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 7618
I20260924 21:12:16.576934 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance
I20260924 21:12:16.577127 6986 external_mini_cluster.cc:1428] Setting key ee40063d5cdb11061654824b41b0032b
I20260924 21:12:16.578189 7596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:16.568753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:47317 (local address 127.6.210.190:41659)
0924 21:12:16.568834 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:16.568836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:16.569026 (+ 190us) server_negotiation.cc:407] Connection header received
0924 21:12:16.570037 (+ 1011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:16.570037 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:16.570050 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:16.570106 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:16.570796 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.571280 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.572383 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.572579 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.572630 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:16.575027 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:16.575029 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:16.575029 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:16.575036 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:16.576678 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:16.577133 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:16.577135 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:16.577135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:16.577198 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:16.577481 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:16.577482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:16.577482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:16.577637 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:16.577718 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:16.578047 (+ 329us) server_negotiation.cc:299] Negotiation successful
0924 21:12:16.578121 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260924 21:12:16.578516 7749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:16.568713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:41659 (local address 127.6.210.129:47317)
0924 21:12:16.568900 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:16.568937 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:16.569937 (+ 1000us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:16.570165 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:16.570169 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:16.570511 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:16.570717 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:16.570723 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.571360 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.571361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:16.572309 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:16.572311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.572650 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.572651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:16.572756 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:16.573284 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:16.573292 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:16.574802 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:16.576759 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:16.576766 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:16.576768 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:16.577063 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:16.577286 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:16.577288 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:16.577288 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:16.577407 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:16.577723 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:16.577729 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:16.577886 (+ 157us) client_negotiation.cc:769] Sending connection context
0924 21:12:16.578003 (+ 117us) client_negotiation.cc:240] Negotiation successful
0924 21:12:16.578179 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":95,"threads_started":1}
I20260924 21:12:16.579613 7747 heartbeater.cc:347] Connected to a master server at 127.6.210.190:41659
I20260924 21:12:16.579733 7747 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:16.579941 7747 heartbeater.cc:511] Master 127.6.210.190:41659 requested a full tablet report, sending...
I20260924 21:12:16.580487 7556 ts_manager.cc:194] Registered new tserver with Master: f1d23c1b9b67452199e6e2c80e899bf7 (127.6.210.129:40859)
I20260924 21:12:16.581073 7556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.129@KRBTEST.COM'} at 127.6.210.129:47317
WARNING: no policy specified for kudu/127.6.210.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
I20260924 21:12:16.603312 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.130:0
--local_ip_for_outbound_sockets=127.6.210.130
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:41659
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:45581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:16.683991 7752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:16.684209 7752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:16.684234 7752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:16.685415 7752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:16.685457 7752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:16.685513 7752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.130
I20260924 21:12:16.687028 7752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:45581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:41659
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.130
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:16.687280 7752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:16.687505 7752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:16.692720 7757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:16.692682 7758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:16.692934 7752 server_base.cc:1034] running on GCE node
W20260924 21:12:16.692678 7760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:16.693259 7752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:16.693480 7752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:16.694636 7752 hybrid_clock.cc:648] HybridClock initialized: now 1790284336694631 us; error 34 us; skew 500 ppm
Sep 24 21:12:16 dist-test-slave-q276 krb5kdc[7511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284336, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:16.697225 7752 init.cc:377] Logged in from keytab as kudu/127.6.210.130@KRBTEST.COM (short username kudu)
I20260924 21:12:16.697777 7752 webserver.cc:460] Webserver started at http://127.6.210.130:33863/ using document root <none> and password file <none>
I20260924 21:12:16.698004 7752 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:16.698071 7752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:16.698235 7752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:16.699321 7752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-1/data/instance:
uuid: "ba10b33baca74aab85a730332a20dde8"
format_stamp: "Formatted at 2026-09-24 21:12:16 on dist-test-slave-q276"
server_key: "18242c32bf7b32fffc9f844e592d12c2"
server_key_iv: "d9ca7c05e03a258f4916d2228e20cbd1"
server_key_version: "encryptionkey@0"
I20260924 21:12:16.699689 7752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance:
uuid: "ba10b33baca74aab85a730332a20dde8"
format_stamp: "Formatted at 2026-09-24 21:12:16 on dist-test-slave-q276"
server_key: "18242c32bf7b32fffc9f844e592d12c2"
server_key_iv: "d9ca7c05e03a258f4916d2228e20cbd1"
server_key_version: "encryptionkey@0"
I20260924 21:12:16.701056 7752 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 21:12:16.701828 7767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:16.702068 7752 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:12:16.702165 7752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-1/wal
uuid: "ba10b33baca74aab85a730332a20dde8"
format_stamp: "Formatted at 2026-09-24 21:12:16 on dist-test-slave-q276"
server_key: "18242c32bf7b32fffc9f844e592d12c2"
server_key_iv: "d9ca7c05e03a258f4916d2228e20cbd1"
server_key_version: "encryptionkey@0"
I20260924 21:12:16.702243 7752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:16.728379 7752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:16.730664 7752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:16.730829 7752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:16.731062 7752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:16.731392 7752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:16.731454 7752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:16.731495 7752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:16.731523 7752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:16.737103 7752 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.130:36657
I20260924 21:12:16.737146 7880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.130:36657 every 8 connection(s)
I20260924 21:12:16.737486 7752 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
I20260924 21:12:16.737996 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 7752
I20260924 21:12:16.738109 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance
I20260924 21:12:16.738305 6986 external_mini_cluster.cc:1428] Setting key 320e0618955118d5d6b5ae64730738e8
Sep 24 21:12:16 dist-test-slave-q276 krb5kdc[7511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284336, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.210.131@KRBTEST.COM; defaulting to no policy
I20260924 21:12:16.748723 7596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:16.738221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:55017 (local address 127.6.210.190:41659)
0924 21:12:16.738298 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:16.738300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:16.738703 (+ 403us) server_negotiation.cc:407] Connection header received
0924 21:12:16.739731 (+ 1028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:16.739732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:16.739744 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:16.739796 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:16.740672 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.741161 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.742803 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.743016 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.743080 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:16.745508 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:16.745510 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:16.745510 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:16.745518 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:16.747143 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:16.747554 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:16.747555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:16.747555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:16.747606 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:16.747990 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:16.747992 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:16.747992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:16.748120 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:16.748185 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:16.748585 (+ 400us) server_negotiation.cc:299] Negotiation successful
0924 21:12:16.748657 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 21:12:16.749099 7883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:16.738225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:41659 (local address 127.6.210.130:55017)
0924 21:12:16.738574 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:16.738619 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:16.739623 (+ 1004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:16.739866 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:16.739870 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:16.740282 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:16.740562 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:16.740568 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.741245 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.741246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:16.742653 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:16.742656 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.743109 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.743110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:16.743278 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:16.743801 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:16.743811 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:16.745418 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:16.747215 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:16.747220 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:16.747222 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:16.747472 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:16.747711 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:16.747712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:16.747712 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:16.747832 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:16.748204 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:16.748209 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:16.748329 (+ 120us) client_negotiation.cc:769] Sending connection context
0924 21:12:16.748559 (+ 230us) client_negotiation.cc:240] Negotiation successful
0924 21:12:16.748698 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":264,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.131@KRBTEST.COM" created.
I20260924 21:12:16.749688 7881 heartbeater.cc:347] Connected to a master server at 127.6.210.190:41659
I20260924 21:12:16.749789 7881 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:16.749989 7881 heartbeater.cc:511] Master 127.6.210.190:41659 requested a full tablet report, sending...
I20260924 21:12:16.750406 7556 ts_manager.cc:194] Registered new tserver with Master: ba10b33baca74aab85a730332a20dde8 (127.6.210.130:36657)
I20260924 21:12:16.750782 7556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.130@KRBTEST.COM'} at 127.6.210.130:55017
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
I20260924 21:12:16.764722 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.131:0
--local_ip_for_outbound_sockets=127.6.210.131
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:41659
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:45581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:16.845950 7886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:16.846192 7886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:16.846231 7886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:16.847419 7886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:16.847486 7886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:16.847533 7886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.131
I20260924 21:12:16.849119 7886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:45581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:41659
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.131
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:16.849382 7886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:16.849622 7886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:16.855058 7894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:16.855036 7892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:16.855177 7891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:16.855417 7886 server_base.cc:1034] running on GCE node
I20260924 21:12:16.855614 7886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:16.855861 7886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:16.857013 7886 hybrid_clock.cc:648] HybridClock initialized: now 1790284336856997 us; error 43 us; skew 500 ppm
Sep 24 21:12:16 dist-test-slave-q276 krb5kdc[7511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284336, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:16.859582 7886 init.cc:377] Logged in from keytab as kudu/127.6.210.131@KRBTEST.COM (short username kudu)
I20260924 21:12:16.860025 7886 webserver.cc:460] Webserver started at http://127.6.210.131:42939/ using document root <none> and password file <none>
I20260924 21:12:16.860270 7886 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:16.860314 7886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:16.860481 7886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:16.861567 7886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-2/data/instance:
uuid: "90c645e763074d098367d1f61cdfbe0f"
format_stamp: "Formatted at 2026-09-24 21:12:16 on dist-test-slave-q276"
server_key: "f5ae599439dec9c71083d189f4c44e4a"
server_key_iv: "fb5d8baafa2c443fe192369fe27043eb"
server_key_version: "encryptionkey@0"
I20260924 21:12:16.861951 7886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance:
uuid: "90c645e763074d098367d1f61cdfbe0f"
format_stamp: "Formatted at 2026-09-24 21:12:16 on dist-test-slave-q276"
server_key: "f5ae599439dec9c71083d189f4c44e4a"
server_key_iv: "fb5d8baafa2c443fe192369fe27043eb"
server_key_version: "encryptionkey@0"
I20260924 21:12:16.863324 7886 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:12:16.864054 7901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:16.864218 7886 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:16.864321 7886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-2/wal
uuid: "90c645e763074d098367d1f61cdfbe0f"
format_stamp: "Formatted at 2026-09-24 21:12:16 on dist-test-slave-q276"
server_key: "f5ae599439dec9c71083d189f4c44e4a"
server_key_iv: "fb5d8baafa2c443fe192369fe27043eb"
server_key_version: "encryptionkey@0"
I20260924 21:12:16.864423 7886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:16.892098 7886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:16.894021 7886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:16.894187 7886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:16.894443 7886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:16.894755 7886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:16.894809 7886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:16.894850 7886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:16.894879 7886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:16.900763 7886 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.131:43555
I20260924 21:12:16.900801 8014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.131:43555 every 8 connection(s)
I20260924 21:12:16.901135 7886 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
Sep 24 21:12:16 dist-test-slave-q276 krb5kdc[7511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284336, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:16.909709 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 7886
I20260924 21:12:16.909843 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance
I20260924 21:12:16.910025 6986 external_mini_cluster.cc:1428] Setting key df8473be13f4e3ed3aa9fba3deee6460
I20260924 21:12:16.910956 7596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:16.901854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:55937 (local address 127.6.210.190:41659)
0924 21:12:16.901919 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:16.901922 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:16.902136 (+ 214us) server_negotiation.cc:407] Connection header received
0924 21:12:16.903116 (+ 980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:16.903117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:16.903127 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:16.903192 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:16.903799 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.904300 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.905345 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.905565 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.905746 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:16.908146 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:16.908148 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:16.908148 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:16.908155 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:16.909395 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:16.909805 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:16.909806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:16.909807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:16.909876 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:16.910212 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:16.910213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:16.910213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:16.910381 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:16.910483 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:16.910763 (+ 280us) server_negotiation.cc:299] Negotiation successful
0924 21:12:16.910849 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 21:12:16.911239 8017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:16.901852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:41659 (local address 127.6.210.131:55937)
0924 21:12:16.902013 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:16.902046 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:16.903014 (+ 968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:16.903261 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:16.903265 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:16.903526 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:16.903717 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:16.903721 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.904374 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.904374 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:16.905247 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:16.905249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.905765 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.905766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:16.905932 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:16.906582 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:16.906590 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:16.908057 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:16.909469 (+ 1412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:16.909474 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:16.909476 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:16.909716 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:16.909964 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:16.909966 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:16.909966 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:16.910113 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:16.910471 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:16.910478 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:16.910616 (+ 138us) client_negotiation.cc:769] Sending connection context
0924 21:12:16.910747 (+ 131us) client_negotiation.cc:240] Negotiation successful
0924 21:12:16.910967 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":84,"threads_started":1}
I20260924 21:12:16.911737 8015 heartbeater.cc:347] Connected to a master server at 127.6.210.190:41659
I20260924 21:12:16.911831 8015 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:16.911968 8015 heartbeater.cc:511] Master 127.6.210.190:41659 requested a full tablet report, sending...
I20260924 21:12:16.912308 7556 ts_manager.cc:194] Registered new tserver with Master: 90c645e763074d098367d1f61cdfbe0f (127.6.210.131:43555)
I20260924 21:12:16.912711 7556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.131@KRBTEST.COM'} at 127.6.210.131:55937
I20260924 21:12:16.921813 6986 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:12:16.929118 7596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:16.923470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51736 (local address 127.6.210.190:41659)
0924 21:12:16.923560 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:16.923562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:16.923645 (+ 83us) server_negotiation.cc:407] Connection header received
0924 21:12:16.923674 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:16.923675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:16.923686 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:16.923760 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:16.924345 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.924826 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.925488 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.925664 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.925721 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:16.926944 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:16.926945 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:16.926945 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:16.926951 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:16.928203 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:16.928509 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:16.928510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:16.928510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:16.928556 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:16.928716 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:16.928716 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:16.928716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:16.928841 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:16.928951 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:16.929001 (+ 50us) server_negotiation.cc:299] Negotiation successful
0924 21:12:16.929055 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260924 21:12:16.931586 7556 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51736:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:12:16.933111 7556 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:12:16.939777 8029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:16.935717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56248 (local address 127.6.210.130:36657)
0924 21:12:16.935897 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:16.935902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:16.935924 (+ 22us) server_negotiation.cc:407] Connection header received
0924 21:12:16.936021 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:16.936023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:16.936038 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:16.936112 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:16.936255 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.937368 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.938663 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.939344 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.939423 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:16.939480 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:16.939652 (+ 172us) server_negotiation.cc:299] Negotiation successful
0924 21:12:16.939695 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":75,"threads_started":1}
I20260924 21:12:16.939987 8026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:16.935369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:40859 (local address 127.0.0.1:45008)
0924 21:12:16.935492 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:16.935524 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:16.935609 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:16.936041 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:16.936044 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:16.936046 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:16.936129 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:16.936130 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.937486 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.937487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:16.938802 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:16.938805 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.939592 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.939593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:16.939662 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:16.939665 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:12:16.939725 (+ 60us) client_negotiation.cc:240] Negotiation successful
0924 21:12:16.939785 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":65,"thread_start_us":75,"threads_started":1}
I20260924 21:12:16.940047 8027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:16.935493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45008 (local address 127.6.210.129:40859)
0924 21:12:16.935772 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:16.935777 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:16.935787 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:12:16.935827 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:16.935830 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:16.935856 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:16.935938 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:16.936220 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.937355 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.938904 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.939505 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.939580 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:16.939635 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:16.939736 (+ 101us) server_negotiation.cc:299] Negotiation successful
0924 21:12:16.939770 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":82,"threads_started":1}
I20260924 21:12:16.940174 8031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:16.935867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38740 (local address 127.6.210.131:43555)
0924 21:12:16.936249 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:16.936253 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:16.936263 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:12:16.936297 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:16.936299 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:16.936315 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:16.936385 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:16.936581 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.937242 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.938889 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.939595 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.939660 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:16.939713 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:16.940035 (+ 322us) server_negotiation.cc:299] Negotiation successful
0924 21:12:16.940086 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":96,"threads_started":1}
I20260924 21:12:16.939998 8030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:16.935596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:36657 (local address 127.0.0.1:56248)
0924 21:12:16.935861 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:16.935871 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:16.935918 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:16.936122 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:16.936123 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:16.936125 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:16.936164 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:16.936165 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.937485 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.937486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:16.938572 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:16.938575 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.939439 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.939441 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:16.939518 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:16.939533 (+ 15us) client_negotiation.cc:769] Sending connection context
0924 21:12:16.939869 (+ 336us) client_negotiation.cc:240] Negotiation successful
0924 21:12:16.939915 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":82,"thread_start_us":71,"threads_started":1}
I20260924 21:12:16.939987 8028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:16.935757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:43555 (local address 127.0.0.1:38740)
0924 21:12:16.935965 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:16.935974 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:16.936015 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:16.936471 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:16.936472 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:16.936473 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:16.936506 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:16.936507 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.937343 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.937344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:16.938802 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:16.938805 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.939746 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.939747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:16.939797 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:16.939800 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:12:16.939846 (+ 46us) client_negotiation.cc:240] Negotiation successful
0924 21:12:16.939887 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":117,"thread_start_us":56,"threads_started":1}
I20260924 21:12:16.940898 7949 tablet_service.cc:1467] Processing CreateTablet for tablet 6016ad2cb0f64f769204ac7665ab6386 (DEFAULT_TABLE table=test-table [id=0d1670e5985e4b21b2e77a010dae34ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:16.940894 7681 tablet_service.cc:1467] Processing CreateTablet for tablet 6016ad2cb0f64f769204ac7665ab6386 (DEFAULT_TABLE table=test-table [id=0d1670e5985e4b21b2e77a010dae34ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:16.941218 7949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6016ad2cb0f64f769204ac7665ab6386. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:16.941234 7681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6016ad2cb0f64f769204ac7665ab6386. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:16.943923 8032 tablet_bootstrap.cc:492] T 6016ad2cb0f64f769204ac7665ab6386 P f1d23c1b9b67452199e6e2c80e899bf7: Bootstrap starting.
I20260924 21:12:16.944363 7815 tablet_service.cc:1467] Processing CreateTablet for tablet 6016ad2cb0f64f769204ac7665ab6386 (DEFAULT_TABLE table=test-table [id=0d1670e5985e4b21b2e77a010dae34ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:16.944684 7815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6016ad2cb0f64f769204ac7665ab6386. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:16.944727 8032 tablet_bootstrap.cc:654] T 6016ad2cb0f64f769204ac7665ab6386 P f1d23c1b9b67452199e6e2c80e899bf7: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:16.945295 8032 log.cc:824] T 6016ad2cb0f64f769204ac7665ab6386 P f1d23c1b9b67452199e6e2c80e899bf7: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:16.946094 8032 tablet_bootstrap.cc:492] T 6016ad2cb0f64f769204ac7665ab6386 P f1d23c1b9b67452199e6e2c80e899bf7: No bootstrap required, opened a new log
I20260924 21:12:16.946175 8032 ts_tablet_manager.cc:1397] T 6016ad2cb0f64f769204ac7665ab6386 P f1d23c1b9b67452199e6e2c80e899bf7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:12:16.948048 8035 tablet_bootstrap.cc:492] T 6016ad2cb0f64f769204ac7665ab6386 P ba10b33baca74aab85a730332a20dde8: Bootstrap starting.
I20260924 21:12:16.948163 8033 tablet_bootstrap.cc:492] T 6016ad2cb0f64f769204ac7665ab6386 P 90c645e763074d098367d1f61cdfbe0f: Bootstrap starting.
I20260924 21:12:16.948324 8032 raft_consensus.cc:348] T 6016ad2cb0f64f769204ac7665ab6386 P f1d23c1b9b67452199e6e2c80e899bf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1d23c1b9b67452199e6e2c80e899bf7" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 40859 } } peers { permanent_uuid: "ba10b33baca74aab85a730332a20dde8" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 36657 } } peers { permanent_uuid: "90c645e763074d098367d1f61cdfbe0f" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 43555 } }
I20260924 21:12:16.948532 8032 raft_consensus.cc:374] T 6016ad2cb0f64f769204ac7665ab6386 P f1d23c1b9b67452199e6e2c80e899bf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:16.948578 8032 raft_consensus.cc:729] T 6016ad2cb0f64f769204ac7665ab6386 P f1d23c1b9b67452199e6e2c80e899bf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1d23c1b9b67452199e6e2c80e899bf7, State: Initialized, Role: FOLLOWER
I20260924 21:12:16.948765 8035 tablet_bootstrap.cc:654] T 6016ad2cb0f64f769204ac7665ab6386 P ba10b33baca74aab85a730332a20dde8: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:16.948704 8032 consensus_queue.cc:260] T 6016ad2cb0f64f769204ac7665ab6386 P f1d23c1b9b67452199e6e2c80e899bf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1d23c1b9b67452199e6e2c80e899bf7" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 40859 } } peers { permanent_uuid: "ba10b33baca74aab85a730332a20dde8" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 36657 } } peers { permanent_uuid: "90c645e763074d098367d1f61cdfbe0f" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 43555 } }
I20260924 21:12:16.948823 8033 tablet_bootstrap.cc:654] T 6016ad2cb0f64f769204ac7665ab6386 P 90c645e763074d098367d1f61cdfbe0f: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:16.949029 8032 ts_tablet_manager.cc:1428] T 6016ad2cb0f64f769204ac7665ab6386 P f1d23c1b9b67452199e6e2c80e899bf7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:12:16.949210 7747 heartbeater.cc:503] Master 127.6.210.190:41659 was elected leader, sending a full tablet report...
I20260924 21:12:16.949522 8033 log.cc:824] T 6016ad2cb0f64f769204ac7665ab6386 P 90c645e763074d098367d1f61cdfbe0f: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:16.949534 8035 log.cc:824] T 6016ad2cb0f64f769204ac7665ab6386 P ba10b33baca74aab85a730332a20dde8: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:16.950259 8033 tablet_bootstrap.cc:492] T 6016ad2cb0f64f769204ac7665ab6386 P 90c645e763074d098367d1f61cdfbe0f: No bootstrap required, opened a new log
I20260924 21:12:16.950350 8033 ts_tablet_manager.cc:1397] T 6016ad2cb0f64f769204ac7665ab6386 P 90c645e763074d098367d1f61cdfbe0f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:12:16.951711 8033 raft_consensus.cc:348] T 6016ad2cb0f64f769204ac7665ab6386 P 90c645e763074d098367d1f61cdfbe0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1d23c1b9b67452199e6e2c80e899bf7" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 40859 } } peers { permanent_uuid: "ba10b33baca74aab85a730332a20dde8" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 36657 } } peers { permanent_uuid: "90c645e763074d098367d1f61cdfbe0f" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 43555 } }
I20260924 21:12:16.951876 8033 raft_consensus.cc:374] T 6016ad2cb0f64f769204ac7665ab6386 P 90c645e763074d098367d1f61cdfbe0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:16.951929 8033 raft_consensus.cc:729] T 6016ad2cb0f64f769204ac7665ab6386 P 90c645e763074d098367d1f61cdfbe0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90c645e763074d098367d1f61cdfbe0f, State: Initialized, Role: FOLLOWER
I20260924 21:12:16.952054 8033 consensus_queue.cc:260] T 6016ad2cb0f64f769204ac7665ab6386 P 90c645e763074d098367d1f61cdfbe0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1d23c1b9b67452199e6e2c80e899bf7" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 40859 } } peers { permanent_uuid: "ba10b33baca74aab85a730332a20dde8" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 36657 } } peers { permanent_uuid: "90c645e763074d098367d1f61cdfbe0f" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 43555 } }
I20260924 21:12:16.952312 8033 ts_tablet_manager.cc:1428] T 6016ad2cb0f64f769204ac7665ab6386 P 90c645e763074d098367d1f61cdfbe0f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260924 21:12:16.952373 8015 heartbeater.cc:503] Master 127.6.210.190:41659 was elected leader, sending a full tablet report...
I20260924 21:12:16.952832 8035 tablet_bootstrap.cc:492] T 6016ad2cb0f64f769204ac7665ab6386 P ba10b33baca74aab85a730332a20dde8: No bootstrap required, opened a new log
I20260924 21:12:16.952939 8035 ts_tablet_manager.cc:1397] T 6016ad2cb0f64f769204ac7665ab6386 P ba10b33baca74aab85a730332a20dde8: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260924 21:12:16.954625 8035 raft_consensus.cc:348] T 6016ad2cb0f64f769204ac7665ab6386 P ba10b33baca74aab85a730332a20dde8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1d23c1b9b67452199e6e2c80e899bf7" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 40859 } } peers { permanent_uuid: "ba10b33baca74aab85a730332a20dde8" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 36657 } } peers { permanent_uuid: "90c645e763074d098367d1f61cdfbe0f" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 43555 } }
I20260924 21:12:16.954763 8035 raft_consensus.cc:374] T 6016ad2cb0f64f769204ac7665ab6386 P ba10b33baca74aab85a730332a20dde8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:16.954787 8035 raft_consensus.cc:729] T 6016ad2cb0f64f769204ac7665ab6386 P ba10b33baca74aab85a730332a20dde8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba10b33baca74aab85a730332a20dde8, State: Initialized, Role: FOLLOWER
I20260924 21:12:16.954873 8035 consensus_queue.cc:260] T 6016ad2cb0f64f769204ac7665ab6386 P ba10b33baca74aab85a730332a20dde8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1d23c1b9b67452199e6e2c80e899bf7" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 40859 } } peers { permanent_uuid: "ba10b33baca74aab85a730332a20dde8" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 36657 } } peers { permanent_uuid: "90c645e763074d098367d1f61cdfbe0f" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 43555 } }
I20260924 21:12:16.955090 8035 ts_tablet_manager.cc:1428] T 6016ad2cb0f64f769204ac7665ab6386 P ba10b33baca74aab85a730332a20dde8: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 21:12:16.955144 7881 heartbeater.cc:503] Master 127.6.210.190:41659 was elected leader, sending a full tablet report...
I20260924 21:12:16.959672 8039 raft_consensus.cc:482] T 6016ad2cb0f64f769204ac7665ab6386 P 90c645e763074d098367d1f61cdfbe0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:12:16.959756 8039 raft_consensus.cc:504] T 6016ad2cb0f64f769204ac7665ab6386 P 90c645e763074d098367d1f61cdfbe0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1d23c1b9b67452199e6e2c80e899bf7" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 40859 } } peers { permanent_uuid: "ba10b33baca74aab85a730332a20dde8" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 36657 } } peers { permanent_uuid: "90c645e763074d098367d1f61cdfbe0f" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 43555 } }
I20260924 21:12:16.959992 8039 leader_election.cc:288] T 6016ad2cb0f64f769204ac7665ab6386 P 90c645e763074d098367d1f61cdfbe0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1d23c1b9b67452199e6e2c80e899bf7 (127.6.210.129:40859), ba10b33baca74aab85a730332a20dde8 (127.6.210.130:36657)
I20260924 21:12:16.963043 8041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:16.960289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:36657 (local address 127.6.210.131:54177)
0924 21:12:16.960431 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:16.960439 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:16.960509 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:16.960620 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:16.960621 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:16.960622 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:16.960685 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:16.960687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.961298 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.961299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:16.962098 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:16.962100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.962736 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.962737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:16.962828 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:16.962833 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 21:12:16.962894 (+ 61us) client_negotiation.cc:240] Negotiation successful
0924 21:12:16.962940 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":3,"thread_start_us":98,"threads_started":1}
I20260924 21:12:16.963066 8029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:16.960337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:54177 (local address 127.6.210.130:36657)
0924 21:12:16.960374 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:16.960376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:16.960500 (+ 124us) server_negotiation.cc:407] Connection header received
0924 21:12:16.960558 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:16.960559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:16.960569 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:16.960619 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:16.960767 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.961215 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.962198 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.962651 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.962737 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:16.962746 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:16.962957 (+ 211us) server_negotiation.cc:299] Negotiation successful
0924 21:12:16.962999 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 21:12:16.963153 8017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:16.960173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:40859 (local address 127.6.210.131:55861)
0924 21:12:16.960240 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:16.960253 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:16.960300 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:16.960472 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:16.960473 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:16.960475 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:16.960553 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:16.960555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.961268 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.961270 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:16.962096 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:16.962098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.962835 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.962836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:16.962925 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:16.962927 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 21:12:16.962976 (+ 49us) client_negotiation.cc:240] Negotiation successful
0924 21:12:16.963010 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":1}
I20260924 21:12:16.963203 8027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:16.960277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:55861 (local address 127.6.210.129:40859)
0924 21:12:16.960340 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:16.960343 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:16.960354 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:12:16.960381 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:16.960382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:16.960395 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:16.960470 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:16.960627 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.961179 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.962194 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.962721 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.962795 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:16.962805 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:16.963072 (+ 267us) server_negotiation.cc:299] Negotiation successful
0924 21:12:16.963116 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 21:12:16.963332 7835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6016ad2cb0f64f769204ac7665ab6386" candidate_uuid: "90c645e763074d098367d1f61cdfbe0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba10b33baca74aab85a730332a20dde8" is_pre_election: true
I20260924 21:12:16.963511 7835 raft_consensus.cc:2454] T 6016ad2cb0f64f769204ac7665ab6386 P ba10b33baca74aab85a730332a20dde8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90c645e763074d098367d1f61cdfbe0f in term 0.
I20260924 21:12:16.963702 7701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6016ad2cb0f64f769204ac7665ab6386" candidate_uuid: "90c645e763074d098367d1f61cdfbe0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1d23c1b9b67452199e6e2c80e899bf7" is_pre_election: true
I20260924 21:12:16.963759 7902 leader_election.cc:302] T 6016ad2cb0f64f769204ac7665ab6386 P 90c645e763074d098367d1f61cdfbe0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90c645e763074d098367d1f61cdfbe0f, ba10b33baca74aab85a730332a20dde8; no voters:
I20260924 21:12:16.963865 7701 raft_consensus.cc:2454] T 6016ad2cb0f64f769204ac7665ab6386 P f1d23c1b9b67452199e6e2c80e899bf7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90c645e763074d098367d1f61cdfbe0f in term 0.
I20260924 21:12:16.963924 8039 raft_consensus.cc:2781] T 6016ad2cb0f64f769204ac7665ab6386 P 90c645e763074d098367d1f61cdfbe0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:12:16.963985 8039 raft_consensus.cc:482] T 6016ad2cb0f64f769204ac7665ab6386 P 90c645e763074d098367d1f61cdfbe0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:12:16.964007 8039 raft_consensus.cc:3037] T 6016ad2cb0f64f769204ac7665ab6386 P 90c645e763074d098367d1f61cdfbe0f [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:16.964664 8039 raft_consensus.cc:504] T 6016ad2cb0f64f769204ac7665ab6386 P 90c645e763074d098367d1f61cdfbe0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1d23c1b9b67452199e6e2c80e899bf7" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 40859 } } peers { permanent_uuid: "ba10b33baca74aab85a730332a20dde8" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 36657 } } peers { permanent_uuid: "90c645e763074d098367d1f61cdfbe0f" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 43555 } }
I20260924 21:12:16.964763 8039 leader_election.cc:288] T 6016ad2cb0f64f769204ac7665ab6386 P 90c645e763074d098367d1f61cdfbe0f [CANDIDATE]: Term 1 election: Requested vote from peers f1d23c1b9b67452199e6e2c80e899bf7 (127.6.210.129:40859), ba10b33baca74aab85a730332a20dde8 (127.6.210.130:36657)
I20260924 21:12:16.964977 7701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6016ad2cb0f64f769204ac7665ab6386" candidate_uuid: "90c645e763074d098367d1f61cdfbe0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1d23c1b9b67452199e6e2c80e899bf7"
I20260924 21:12:16.965046 7701 raft_consensus.cc:3037] T 6016ad2cb0f64f769204ac7665ab6386 P f1d23c1b9b67452199e6e2c80e899bf7 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:16.966140 7701 raft_consensus.cc:2454] T 6016ad2cb0f64f769204ac7665ab6386 P f1d23c1b9b67452199e6e2c80e899bf7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90c645e763074d098367d1f61cdfbe0f in term 1.
I20260924 21:12:16.966257 7835 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6016ad2cb0f64f769204ac7665ab6386" candidate_uuid: "90c645e763074d098367d1f61cdfbe0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba10b33baca74aab85a730332a20dde8"
I20260924 21:12:16.966315 7902 leader_election.cc:302] T 6016ad2cb0f64f769204ac7665ab6386 P 90c645e763074d098367d1f61cdfbe0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90c645e763074d098367d1f61cdfbe0f, f1d23c1b9b67452199e6e2c80e899bf7; no voters:
I20260924 21:12:16.966378 7835 raft_consensus.cc:3037] T 6016ad2cb0f64f769204ac7665ab6386 P ba10b33baca74aab85a730332a20dde8 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:16.966403 8039 raft_consensus.cc:2781] T 6016ad2cb0f64f769204ac7665ab6386 P 90c645e763074d098367d1f61cdfbe0f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:16.966580 8039 raft_consensus.cc:686] T 6016ad2cb0f64f769204ac7665ab6386 P 90c645e763074d098367d1f61cdfbe0f [term 1 LEADER]: Becoming Leader. State: Replica: 90c645e763074d098367d1f61cdfbe0f, State: Running, Role: LEADER
I20260924 21:12:16.966678 8039 consensus_queue.cc:237] T 6016ad2cb0f64f769204ac7665ab6386 P 90c645e763074d098367d1f61cdfbe0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1d23c1b9b67452199e6e2c80e899bf7" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 40859 } } peers { permanent_uuid: "ba10b33baca74aab85a730332a20dde8" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 36657 } } peers { permanent_uuid: "90c645e763074d098367d1f61cdfbe0f" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 43555 } }
I20260924 21:12:16.967228 7835 raft_consensus.cc:2454] T 6016ad2cb0f64f769204ac7665ab6386 P ba10b33baca74aab85a730332a20dde8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90c645e763074d098367d1f61cdfbe0f in term 1.
I20260924 21:12:16.967506 7556 catalog_manager.cc:5495] T 6016ad2cb0f64f769204ac7665ab6386 P 90c645e763074d098367d1f61cdfbe0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 90c645e763074d098367d1f61cdfbe0f (127.6.210.131). New cstate: current_term: 1 leader_uuid: "90c645e763074d098367d1f61cdfbe0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1d23c1b9b67452199e6e2c80e899bf7" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 40859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba10b33baca74aab85a730332a20dde8" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 36657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90c645e763074d098367d1f61cdfbe0f" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 43555 } health_report { overall_health: HEALTHY } } }
Sep 24 21:12:16 dist-test-slave-q276 krb5kdc[7511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.129@KRBTEST.COM
I20260924 21:12:16.979182 8027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:16.971415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45012 (local address 127.6.210.129:40859)
0924 21:12:16.971500 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:16.971502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:16.971619 (+ 117us) server_negotiation.cc:407] Connection header received
0924 21:12:16.971695 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:16.971696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:16.971705 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:16.971784 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:16.972323 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.972783 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.973486 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.973680 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.973753 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:16.976335 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:16.976343 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:16.976343 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:16.976350 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:16.978012 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:16.978463 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:16.978466 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:16.978466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:16.978521 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:16.978721 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:16.978722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:16.978722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:16.978837 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:16.978992 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:16.979052 (+ 60us) server_negotiation.cc:299] Negotiation successful
0924 21:12:16.979117 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260924 21:12:16.988673 7882 tablet.cc:2369] T 6016ad2cb0f64f769204ac7665ab6386 P ba10b33baca74aab85a730332a20dde8: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 24 21:12:16 dist-test-slave-q276 krb5kdc[7511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284336, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 21:12:16 dist-test-slave-q276 krb5kdc[7511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284336, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.210.129@KRBTEST.COM
I20260924 21:12:17.001564 8027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:16.994478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45020 (local address 127.6.210.129:40859)
0924 21:12:16.994538 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:16.994540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:16.994650 (+ 110us) server_negotiation.cc:407] Connection header received
0924 21:12:16.994736 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:16.994737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:16.994748 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:16.994827 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:16.995391 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.995869 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.996602 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:16.996797 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:16.996880 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:16.999212 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:16.999213 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:16.999214 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:16.999221 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:17.000499 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:17.000848 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:17.000849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:17.000849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:17.000897 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:17.001113 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:17.001113 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:17.001114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:17.001204 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:17.001274 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:17.001434 (+ 160us) server_negotiation.cc:299] Negotiation successful
0924 21:12:17.001497 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
Sep 24 21:12:17 dist-test-slave-q276 krb5kdc[7511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 21:12:17 dist-test-slave-q276 krb5kdc[7511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.129@KRBTEST.COM
I20260924 21:12:17.024542 8027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:17.017277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45036 (local address 127.6.210.129:40859)
0924 21:12:17.017354 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:17.017356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:17.017399 (+ 43us) server_negotiation.cc:407] Connection header received
0924 21:12:17.017452 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:17.017453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:17.017464 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:17.017514 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:17.018325 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:17.018809 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:17.019541 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:17.019717 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:17.019767 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:17.022045 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:17.022047 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:17.022047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:17.022054 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:17.023617 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:17.023939 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:17.023940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:17.023940 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:17.023989 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:17.024137 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:17.024137 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:17.024138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:17.024217 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:17.024282 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:17.024369 (+ 87us) server_negotiation.cc:299] Negotiation successful
0924 21:12:17.024427 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260924 21:12:17.024962 7661 tablet_service.cc:2908] Rejecting scan request for tablet 6016ad2cb0f64f769204ac7665ab6386: Uninitialized: clean time has not yet been initialized
I20260924 21:12:17.025712 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 7618
I20260924 21:12:17.030339 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 7752
I20260924 21:12:17.035152 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 7886
I20260924 21:12:17.039388 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 7524
2026-09-24T21:12:17Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2025 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:12:17 dist-test-slave-q276 krb5kdc[8056](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:12:17 dist-test-slave-q276 krb5kdc[8056](info): set up 2 sockets
Sep 24 21:12:17 dist-test-slave-q276 krb5kdc[8056](info): commencing operation
krb5kdc: starting...
W20260924 21:12:18.095054 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.002s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:12:18 dist-test-slave-q276 krb5kdc[8056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284338, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:12:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:12:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.210.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
I20260924 21:12:18.213402 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:39053
--webserver_interface=127.6.210.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:46877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:39053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:18.293071 8069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:18.293321 8069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:18.293358 8069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:18.294592 8069 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:12:18.294651 8069 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260924 21:12:18.294674 8069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:18.294708 8069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:18.294741 8069 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:12:18.294785 8069 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:12:18.296312 8069 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:46877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:39053
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:39053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:18.296589 8069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:18.296815 8069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:18.301983 8075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:18.301983 8077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:18.302093 8074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:18.302146 8069 server_base.cc:1034] running on GCE node
I20260924 21:12:18.302630 8069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:18.302922 8069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:18.304090 8069 hybrid_clock.cc:648] HybridClock initialized: now 1790284338304067 us; error 41 us; skew 500 ppm
Sep 24 21:12:18 dist-test-slave-q276 krb5kdc[8056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284338, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:18.306849 8069 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:12:18.307281 8069 webserver.cc:460] Webserver started at http://127.6.210.190:37823/ using document root <none> and password file <none>
I20260924 21:12:18.307505 8069 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:18.307586 8069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:18.307718 8069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:18.308743 8069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "271d3ea1dc374c169abd29787d688cb9"
format_stamp: "Formatted at 2026-09-24 21:12:18 on dist-test-slave-q276"
server_key: "d078f75c77e920e10ebfffc334ddffaf"
server_key_iv: "7502ac79f3ff4064658517e873ab35bf"
server_key_version: "encryptionkey@0"
I20260924 21:12:18.309048 8069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "271d3ea1dc374c169abd29787d688cb9"
format_stamp: "Formatted at 2026-09-24 21:12:18 on dist-test-slave-q276"
server_key: "d078f75c77e920e10ebfffc334ddffaf"
server_key_iv: "7502ac79f3ff4064658517e873ab35bf"
server_key_version: "encryptionkey@0"
I20260924 21:12:18.310310 8069 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:12:18.311070 8084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:18.311256 8069 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:18.311359 8069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "271d3ea1dc374c169abd29787d688cb9"
format_stamp: "Formatted at 2026-09-24 21:12:18 on dist-test-slave-q276"
server_key: "d078f75c77e920e10ebfffc334ddffaf"
server_key_iv: "7502ac79f3ff4064658517e873ab35bf"
server_key_version: "encryptionkey@0"
I20260924 21:12:18.311457 8069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:18.323218 8069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:18.325278 8069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:18.325459 8069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:18.329880 8069 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:39053
I20260924 21:12:18.329978 8146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:39053 every 8 connection(s)
I20260924 21:12:18.330327 8069 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:12:18.331025 8147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:18.333235 8147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 271d3ea1dc374c169abd29787d688cb9: Bootstrap starting.
I20260924 21:12:18.333813 8147 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 271d3ea1dc374c169abd29787d688cb9: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:18.334100 8147 log.cc:824] T 00000000000000000000000000000000 P 271d3ea1dc374c169abd29787d688cb9: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:18.334829 8147 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 271d3ea1dc374c169abd29787d688cb9: No bootstrap required, opened a new log
I20260924 21:12:18.336935 8147 raft_consensus.cc:348] T 00000000000000000000000000000000 P 271d3ea1dc374c169abd29787d688cb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271d3ea1dc374c169abd29787d688cb9" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39053 } }
I20260924 21:12:18.337069 8147 raft_consensus.cc:374] T 00000000000000000000000000000000 P 271d3ea1dc374c169abd29787d688cb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:18.337093 8147 raft_consensus.cc:729] T 00000000000000000000000000000000 P 271d3ea1dc374c169abd29787d688cb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 271d3ea1dc374c169abd29787d688cb9, State: Initialized, Role: FOLLOWER
I20260924 21:12:18.337260 8147 consensus_queue.cc:260] T 00000000000000000000000000000000 P 271d3ea1dc374c169abd29787d688cb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271d3ea1dc374c169abd29787d688cb9" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39053 } }
I20260924 21:12:18.337320 8147 raft_consensus.cc:388] T 00000000000000000000000000000000 P 271d3ea1dc374c169abd29787d688cb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:12:18.337342 8147 raft_consensus.cc:482] T 00000000000000000000000000000000 P 271d3ea1dc374c169abd29787d688cb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:12:18.337412 8147 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 271d3ea1dc374c169abd29787d688cb9 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:18.337970 8147 raft_consensus.cc:504] T 00000000000000000000000000000000 P 271d3ea1dc374c169abd29787d688cb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271d3ea1dc374c169abd29787d688cb9" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39053 } }
I20260924 21:12:18.338084 8147 leader_election.cc:302] T 00000000000000000000000000000000 P 271d3ea1dc374c169abd29787d688cb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 271d3ea1dc374c169abd29787d688cb9; no voters:
I20260924 21:12:18.338157 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 8069
I20260924 21:12:18.338246 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:12:18.338282 8147 leader_election.cc:288] T 00000000000000000000000000000000 P 271d3ea1dc374c169abd29787d688cb9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:12:18.338347 8150 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 271d3ea1dc374c169abd29787d688cb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:18.338455 6986 external_mini_cluster.cc:1428] Setting key fa52dd765dc30acb2495d5e91ef7d585
I20260924 21:12:18.338539 8147 sys_catalog.cc:564] T 00000000000000000000000000000000 P 271d3ea1dc374c169abd29787d688cb9 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:12:18.338546 8150 raft_consensus.cc:686] T 00000000000000000000000000000000 P 271d3ea1dc374c169abd29787d688cb9 [term 1 LEADER]: Becoming Leader. State: Replica: 271d3ea1dc374c169abd29787d688cb9, State: Running, Role: LEADER
I20260924 21:12:18.338663 8150 consensus_queue.cc:237] T 00000000000000000000000000000000 P 271d3ea1dc374c169abd29787d688cb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271d3ea1dc374c169abd29787d688cb9" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39053 } }
I20260924 21:12:18.339167 8151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 271d3ea1dc374c169abd29787d688cb9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "271d3ea1dc374c169abd29787d688cb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271d3ea1dc374c169abd29787d688cb9" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39053 } } }
I20260924 21:12:18.339182 8152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 271d3ea1dc374c169abd29787d688cb9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 271d3ea1dc374c169abd29787d688cb9. Latest consensus state: current_term: 1 leader_uuid: "271d3ea1dc374c169abd29787d688cb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "271d3ea1dc374c169abd29787d688cb9" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39053 } } }
I20260924 21:12:18.339285 8151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 271d3ea1dc374c169abd29787d688cb9 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:18.339291 8152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 271d3ea1dc374c169abd29787d688cb9 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:18.340245 8156 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:12:18.340763 8156 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:12:18.342307 8156 catalog_manager.cc:1347] Generated new cluster ID: aebaf7152c574ae193e49e70d5b1e2f1
I20260924 21:12:18.342358 8156 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 21:12:18 dist-test-slave-q276 krb5kdc[8056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284338, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:18.349895 8154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:18.338954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34432 (local address 127.6.210.190:39053)
0924 21:12:18.339138 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:18.339148 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:18.339185 (+ 37us) server_negotiation.cc:407] Connection header received
0924 21:12:18.339301 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:18.339309 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:18.339371 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:18.340065 (+ 694us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:18.340424 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.341714 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.342746 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.343343 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.343430 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:18.346106 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:18.346110 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:18.346110 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:18.346121 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:18.348059 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:18.348432 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:18.348435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:18.348436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:18.348528 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:18.348701 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:18.348701 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:18.348701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:18.348973 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:18.349099 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:18.349537 (+ 438us) server_negotiation.cc:299] Negotiation successful
0924 21:12:18.349659 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":75,"threads_started":1}
I20260924 21:12:18.352731 8156 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:12:18.353199 8156 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:12:18.361243 8156 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 271d3ea1dc374c169abd29787d688cb9: Generated new TSK 0
I20260924 21:12:18.361444 8156 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.210.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
I20260924 21:12:18.389660 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.129:0
--local_ip_for_outbound_sockets=127.6.210.129
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:39053
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:46877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:18.471773 8173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:18.472016 8173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:18.472059 8173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:18.473259 8173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:18.473326 8173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:18.473361 8173 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 21:12:18.473402 8173 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.129
I20260924 21:12:18.474879 8173 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:46877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.210.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.210.190:39053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.129
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:18.475150 8173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:18.475389 8173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:18.480513 8178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:18.480540 8181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:18.480557 8173 server_base.cc:1034] running on GCE node
W20260924 21:12:18.480525 8179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:18.481035 8173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:18.481304 8173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:18.482489 8173 hybrid_clock.cc:648] HybridClock initialized: now 1790284338482484 us; error 53 us; skew 500 ppm
Sep 24 21:12:18 dist-test-slave-q276 krb5kdc[8056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284338, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:18.484900 8173 init.cc:377] Logged in from keytab as kudu/127.6.210.129@KRBTEST.COM (short username kudu)
I20260924 21:12:18.485327 8173 webserver.cc:460] Webserver started at http://127.6.210.129:46225/ using document root <none> and password file <none>
I20260924 21:12:18.485570 8173 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:18.485639 8173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:18.485781 8173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:18.486896 8173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-0/data/instance:
uuid: "205cc24777c74b22b462a666e3216372"
format_stamp: "Formatted at 2026-09-24 21:12:18 on dist-test-slave-q276"
server_key: "2ba8ef9c944b7898b050a38c3bd86a3e"
server_key_iv: "367c8527f2dd9bb7f4372d14406efc27"
server_key_version: "encryptionkey@0"
I20260924 21:12:18.487254 8173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance:
uuid: "205cc24777c74b22b462a666e3216372"
format_stamp: "Formatted at 2026-09-24 21:12:18 on dist-test-slave-q276"
server_key: "2ba8ef9c944b7898b050a38c3bd86a3e"
server_key_iv: "367c8527f2dd9bb7f4372d14406efc27"
server_key_version: "encryptionkey@0"
I20260924 21:12:18.488569 8173 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:12:18.489300 8188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:18.489574 8173 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:18.489676 8173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-0/wal
uuid: "205cc24777c74b22b462a666e3216372"
format_stamp: "Formatted at 2026-09-24 21:12:18 on dist-test-slave-q276"
server_key: "2ba8ef9c944b7898b050a38c3bd86a3e"
server_key_iv: "367c8527f2dd9bb7f4372d14406efc27"
server_key_version: "encryptionkey@0"
I20260924 21:12:18.489746 8173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:18.502334 8173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:18.504439 8173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:18.504631 8173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:18.505151 8173 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:18.505218 8173 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:18.505265 8173 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:18.505293 8173 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:18.512496 8173 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.129:42409
I20260924 21:12:18.512567 8303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.129:42409 every 8 connection(s)
Sep 24 21:12:18 dist-test-slave-q276 krb5kdc[8056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284338, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:18.512902 8173 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
I20260924 21:12:18.514761 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 8173
I20260924 21:12:18.514875 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance
I20260924 21:12:18.515064 6986 external_mini_cluster.cc:1428] Setting key 0182c5b6be6152b29a7a89a611f24014
I20260924 21:12:18.516286 8154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:18.505702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:60441 (local address 127.6.210.190:39053)
0924 21:12:18.505779 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:18.505781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:18.506017 (+ 236us) server_negotiation.cc:407] Connection header received
0924 21:12:18.506772 (+ 755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:18.506773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:18.506783 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:18.506859 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:18.507814 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.508633 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.510334 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.510548 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.510604 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:18.513406 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:18.513407 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:18.513408 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:18.513414 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:18.514742 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:18.515237 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:18.515238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:18.515239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:18.515306 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:18.515611 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:18.515612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:18.515613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:18.515751 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:18.515832 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:18.516158 (+ 326us) server_negotiation.cc:299] Negotiation successful
0924 21:12:18.516225 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260924 21:12:18.516638 8197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:18.505673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39053 (local address 127.6.210.129:60441)
0924 21:12:18.505857 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:18.505907 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:18.506657 (+ 750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:18.506958 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:18.506972 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:18.507376 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:18.507707 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:18.507715 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.508735 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.508736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:18.510230 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:18.510233 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.510629 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.510630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:18.510807 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:18.511407 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:18.511419 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:18.513314 (+ 1895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:18.514819 (+ 1505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:18.514826 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:18.514828 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:18.515136 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:18.515384 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:18.515385 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:18.515386 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:18.515543 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:18.515834 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:18.515840 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:18.516019 (+ 179us) client_negotiation.cc:769] Sending connection context
0924 21:12:18.516137 (+ 118us) client_negotiation.cc:240] Negotiation successful
0924 21:12:18.516320 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":86,"threads_started":1}
I20260924 21:12:18.517488 8304 heartbeater.cc:347] Connected to a master server at 127.6.210.190:39053
I20260924 21:12:18.517570 8304 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:18.517757 8304 heartbeater.cc:511] Master 127.6.210.190:39053 requested a full tablet report, sending...
I20260924 21:12:18.518337 8100 ts_manager.cc:194] Registered new tserver with Master: 205cc24777c74b22b462a666e3216372 (127.6.210.129:42409)
I20260924 21:12:18.519019 8100 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.129@KRBTEST.COM'} at 127.6.210.129:60441
WARNING: no policy specified for kudu/127.6.210.130@KRBTEST.COM; defaulting to no policy
I20260924 21:12:18.524679 8311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:18.518553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39053 (local address 127.6.210.129:60317)
0924 21:12:18.518683 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:18.518694 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:18.518745 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:18.519401 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:18.519402 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:18.519582 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:18.519642 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:18.519643 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.520289 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.520290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:18.520779 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:18.520781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.521178 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.521179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:18.521256 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:18.521722 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:18.521731 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:18.522010 (+ 279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:18.523631 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:18.523633 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:18.523633 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:18.523803 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:18.524032 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:18.524032 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:18.524033 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:18.524096 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:18.524336 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:18.524337 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:18.524383 (+ 46us) client_negotiation.cc:769] Sending connection context
0924 21:12:18.524470 (+ 87us) client_negotiation.cc:240] Negotiation successful
0924 21:12:18.524573 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":72,"threads_started":1}
I20260924 21:12:18.524724 8154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:18.519214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:60317 (local address 127.6.210.190:39053)
0924 21:12:18.519266 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:18.519268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:18.519275 (+ 7us) server_negotiation.cc:407] Connection header received
0924 21:12:18.519294 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:18.519295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:18.519303 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:18.519359 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:18.519720 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.520190 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.520889 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.521095 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.521178 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:18.522117 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:18.522119 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:18.522119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:18.522126 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:18.523532 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:18.523900 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:18.523901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:18.523901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:18.523960 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:18.524156 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:18.524157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:18.524158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:18.524271 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:18.524346 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:18.524495 (+ 149us) server_negotiation.cc:299] Negotiation successful
0924 21:12:18.524573 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
I20260924 21:12:18.542233 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.130:0
--local_ip_for_outbound_sockets=127.6.210.130
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:39053
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:46877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:18.622195 8313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:18.622431 8313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:18.622455 8313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:18.623591 8313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:18.623629 8313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:18.623649 8313 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 21:12:18.623667 8313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.130
I20260924 21:12:18.625006 8313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:46877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.210.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.210.190:39053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.130
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:18.625195 8313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:18.625366 8313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:18.630558 8319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:18.630585 8313 server_base.cc:1034] running on GCE node
W20260924 21:12:18.630568 8321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:18.630592 8318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:18.631024 8313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:18.631294 8313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:18.632459 8313 hybrid_clock.cc:648] HybridClock initialized: now 1790284338632434 us; error 42 us; skew 500 ppm
Sep 24 21:12:18 dist-test-slave-q276 krb5kdc[8056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284338, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:18.635030 8313 init.cc:377] Logged in from keytab as kudu/127.6.210.130@KRBTEST.COM (short username kudu)
I20260924 21:12:18.635530 8313 webserver.cc:460] Webserver started at http://127.6.210.130:36291/ using document root <none> and password file <none>
I20260924 21:12:18.635767 8313 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:18.635836 8313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:18.635987 8313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:18.637084 8313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-1/data/instance:
uuid: "82d3aabdbec945738a0b35a8b2db05ec"
format_stamp: "Formatted at 2026-09-24 21:12:18 on dist-test-slave-q276"
server_key: "2955a09507e93b57c919129f200cfd2e"
server_key_iv: "c968c6aa77c7927f2f68db0a1c33ca6f"
server_key_version: "encryptionkey@0"
I20260924 21:12:18.637442 8313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance:
uuid: "82d3aabdbec945738a0b35a8b2db05ec"
format_stamp: "Formatted at 2026-09-24 21:12:18 on dist-test-slave-q276"
server_key: "2955a09507e93b57c919129f200cfd2e"
server_key_iv: "c968c6aa77c7927f2f68db0a1c33ca6f"
server_key_version: "encryptionkey@0"
I20260924 21:12:18.638860 8313 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:12:18.639643 8328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:18.639838 8313 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:12:18.639936 8313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-1/wal
uuid: "82d3aabdbec945738a0b35a8b2db05ec"
format_stamp: "Formatted at 2026-09-24 21:12:18 on dist-test-slave-q276"
server_key: "2955a09507e93b57c919129f200cfd2e"
server_key_iv: "c968c6aa77c7927f2f68db0a1c33ca6f"
server_key_version: "encryptionkey@0"
I20260924 21:12:18.640026 8313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:18.659255 8313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:18.661576 8313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:18.661777 8313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:18.662422 8313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:18.662492 8313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:18.662530 8313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:18.662560 8313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:18.670495 8313 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.130:34611
I20260924 21:12:18.670589 8443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.130:34611 every 8 connection(s)
Sep 24 21:12:18 dist-test-slave-q276 krb5kdc[8056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284338, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:18.670936 8313 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
I20260924 21:12:18.673841 8154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:18.662846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:42901 (local address 127.6.210.190:39053)
0924 21:12:18.662915 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:18.662918 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:18.663236 (+ 318us) server_negotiation.cc:407] Connection header received
0924 21:12:18.664041 (+ 805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:18.664042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:18.664054 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:18.664138 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:18.665169 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.665721 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.667637 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.667849 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.667907 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:18.671312 (+ 3405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:18.671314 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:18.671314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:18.671322 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:18.672723 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:18.673100 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:18.673101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:18.673102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:18.673149 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:18.673330 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:18.673330 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:18.673330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:18.673423 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:18.673481 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:18.673722 (+ 241us) server_negotiation.cc:299] Negotiation successful
0924 21:12:18.673779 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 21:12:18.674075 8337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:18.662846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39053 (local address 127.6.210.130:42901)
0924 21:12:18.663045 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:18.663099 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:18.663901 (+ 802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:18.664241 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:18.664247 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:18.664692 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:18.665058 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:18.665066 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.665807 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.665808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:18.667516 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:18.667520 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.667930 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.667931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:18.668108 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:18.669341 (+ 1233us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:18.669354 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:18.671183 (+ 1829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:18.672802 (+ 1619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:18.672806 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:18.672808 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:18.673033 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:18.673190 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:18.673191 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:18.673191 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:18.673279 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:18.673492 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:18.673497 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:18.673614 (+ 117us) client_negotiation.cc:769] Sending connection context
0924 21:12:18.673699 (+ 85us) client_negotiation.cc:240] Negotiation successful
0924 21:12:18.673822 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":105,"threads_started":1}
I20260924 21:12:18.675817 8444 heartbeater.cc:347] Connected to a master server at 127.6.210.190:39053
I20260924 21:12:18.675926 8444 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:18.676119 8444 heartbeater.cc:511] Master 127.6.210.190:39053 requested a full tablet report, sending...
I20260924 21:12:18.676442 8100 ts_manager.cc:194] Registered new tserver with Master: 82d3aabdbec945738a0b35a8b2db05ec (127.6.210.130:34611)
I20260924 21:12:18.676934 8100 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.130@KRBTEST.COM'} at 127.6.210.130:42901
I20260924 21:12:18.677122 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 8313
I20260924 21:12:18.677213 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance
I20260924 21:12:18.677381 6986 external_mini_cluster.cc:1428] Setting key 037f8abf2dc3117de33338b50a26d704
I20260924 21:12:18.681807 8154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:18.675633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:56659 (local address 127.6.210.190:39053)
0924 21:12:18.675722 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:18.675725 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:18.675790 (+ 65us) server_negotiation.cc:407] Connection header received
0924 21:12:18.675853 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:18.675854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:18.675865 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:18.675934 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:18.676263 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.677007 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.677823 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.678159 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.678239 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:18.679253 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:18.679254 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:18.679254 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:18.679260 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:18.680588 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:18.681001 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:18.681002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:18.681002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:18.681051 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:18.681283 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:18.681285 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:18.681285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:18.681458 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:18.681519 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:18.681695 (+ 176us) server_negotiation.cc:299] Negotiation successful
0924 21:12:18.681750 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260924 21:12:18.681905 8450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:18.675585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39053 (local address 127.6.210.130:56659)
0924 21:12:18.675731 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:18.675742 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:18.675794 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:18.675936 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:18.675937 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:18.676112 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:18.676192 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:18.676193 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.677116 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.677117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:18.677740 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:18.677742 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.678241 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.678242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:18.678340 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:18.678783 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:18.678788 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:18.679165 (+ 377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:18.680671 (+ 1506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:18.680672 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:18.680672 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:18.680900 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:18.681117 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:18.681118 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:18.681118 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:18.681189 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:18.681533 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:18.681534 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:18.681593 (+ 59us) client_negotiation.cc:769] Sending connection context
0924 21:12:18.681669 (+ 76us) client_negotiation.cc:240] Negotiation successful
0924 21:12:18.681774 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":58,"threads_started":1}
WARNING: no policy specified for kudu/127.6.210.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
I20260924 21:12:18.701306 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.131:0
--local_ip_for_outbound_sockets=127.6.210.131
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:39053
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:46877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:18.781616 8453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:18.781826 8453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:18.781850 8453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:18.782995 8453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:18.783030 8453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:18.783051 8453 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 21:12:18.783067 8453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.131
I20260924 21:12:18.784401 8453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:46877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.210.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.210.190:39053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.131
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:18.784588 8453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:18.784761 8453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:18.789703 8458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:18.789676 8461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:18.789680 8459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:18.789728 8453 server_base.cc:1034] running on GCE node
I20260924 21:12:18.790086 8453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:18.790331 8453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:18.791509 8453 hybrid_clock.cc:648] HybridClock initialized: now 1790284338791494 us; error 32 us; skew 500 ppm
Sep 24 21:12:18 dist-test-slave-q276 krb5kdc[8056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284338, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:18.793937 8453 init.cc:377] Logged in from keytab as kudu/127.6.210.131@KRBTEST.COM (short username kudu)
I20260924 21:12:18.794440 8453 webserver.cc:460] Webserver started at http://127.6.210.131:37397/ using document root <none> and password file <none>
I20260924 21:12:18.794672 8453 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:18.794740 8453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:18.794924 8453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:18.796049 8453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-2/data/instance:
uuid: "e886851288674ad4bf7ba99c9d769e3d"
format_stamp: "Formatted at 2026-09-24 21:12:18 on dist-test-slave-q276"
server_key: "cebb1dee9ce65e26a9f487892096cb18"
server_key_iv: "7460699419aa370b759d70a570d34b7d"
server_key_version: "encryptionkey@0"
I20260924 21:12:18.796411 8453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance:
uuid: "e886851288674ad4bf7ba99c9d769e3d"
format_stamp: "Formatted at 2026-09-24 21:12:18 on dist-test-slave-q276"
server_key: "cebb1dee9ce65e26a9f487892096cb18"
server_key_iv: "7460699419aa370b759d70a570d34b7d"
server_key_version: "encryptionkey@0"
I20260924 21:12:18.797709 8453 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 21:12:18.798735 8468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:18.798906 8453 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:18.799005 8453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-2/wal
uuid: "e886851288674ad4bf7ba99c9d769e3d"
format_stamp: "Formatted at 2026-09-24 21:12:18 on dist-test-slave-q276"
server_key: "cebb1dee9ce65e26a9f487892096cb18"
server_key_iv: "7460699419aa370b759d70a570d34b7d"
server_key_version: "encryptionkey@0"
I20260924 21:12:18.799090 8453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:18.823961 8453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:18.825909 8453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:18.826081 8453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:18.826687 8453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:18.826766 8453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:18.826803 8453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:18.826826 8453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 21:12:18 dist-test-slave-q276 krb5kdc[8056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284338, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:18.833746 8453 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.131:40047
I20260924 21:12:18.833895 8583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.131:40047 every 8 connection(s)
I20260924 21:12:18.834179 8453 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
I20260924 21:12:18.835855 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 8453
I20260924 21:12:18.836007 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance
I20260924 21:12:18.836184 6986 external_mini_cluster.cc:1428] Setting key e49137c4b6cc740c83deada30abce132
I20260924 21:12:18.836423 8154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:18.827222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:47003 (local address 127.6.210.190:39053)
0924 21:12:18.827293 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:18.827295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:18.827496 (+ 201us) server_negotiation.cc:407] Connection header received
0924 21:12:18.828045 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:18.828046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:18.828062 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:18.828148 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:18.829044 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.829871 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.831027 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.831239 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.831327 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:18.833498 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:18.833500 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:18.833501 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:18.833509 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:18.835177 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:18.835525 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:18.835525 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:18.835525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:18.835574 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:18.835793 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:18.835794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:18.835794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:18.835917 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:18.836000 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:18.836254 (+ 254us) server_negotiation.cc:299] Negotiation successful
0924 21:12:18.836331 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 21:12:18.836608 8477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:18.827204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39053 (local address 127.6.210.131:47003)
0924 21:12:18.827376 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:18.827416 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:18.827906 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:18.828239 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:18.828243 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:18.828649 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:18.828940 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:18.828946 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.829986 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.829988 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:18.830930 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:18.830933 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.831347 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.831347 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:18.831479 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:18.832022 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:18.832030 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:18.833397 (+ 1367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:18.835247 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:18.835251 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:18.835252 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:18.835456 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:18.835622 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:18.835623 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:18.835623 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:18.835705 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:18.835990 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:18.835995 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:18.836123 (+ 128us) client_negotiation.cc:769] Sending connection context
0924 21:12:18.836231 (+ 108us) client_negotiation.cc:240] Negotiation successful
0924 21:12:18.836385 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":105,"threads_started":1}
I20260924 21:12:18.837057 8584 heartbeater.cc:347] Connected to a master server at 127.6.210.190:39053
I20260924 21:12:18.837145 8584 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:18.837340 8584 heartbeater.cc:511] Master 127.6.210.190:39053 requested a full tablet report, sending...
I20260924 21:12:18.837754 8100 ts_manager.cc:194] Registered new tserver with Master: e886851288674ad4bf7ba99c9d769e3d (127.6.210.131:40047)
I20260924 21:12:18.838203 8100 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.131@KRBTEST.COM'} at 127.6.210.131:47003
I20260924 21:12:18.844201 8154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:18.838434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:53721 (local address 127.6.210.190:39053)
0924 21:12:18.838475 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:18.838477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:18.838580 (+ 103us) server_negotiation.cc:407] Connection header received
0924 21:12:18.839082 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:18.839083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:18.839094 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:18.839169 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:18.839539 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.840003 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.840741 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.840935 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.841014 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:18.841806 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:18.841807 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:18.841807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:18.841813 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:18.843171 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:18.843473 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:18.843474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:18.843474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:18.843530 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:18.843725 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:18.843726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:18.843727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:18.843844 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:18.843900 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:18.844053 (+ 153us) server_negotiation.cc:299] Negotiation successful
0924 21:12:18.844132 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 21:12:18.844200 8590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:18.838303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39053 (local address 127.6.210.131:53721)
0924 21:12:18.838513 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:18.838523 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:18.838578 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:18.839176 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:18.839176 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:18.839363 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:18.839415 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:18.839417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.840093 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.840094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:18.840646 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:18.840648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.841039 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.841040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:18.841095 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:18.841477 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:18.841480 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:18.841734 (+ 254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:18.843235 (+ 1501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:18.843236 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:18.843236 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:18.843382 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:18.843595 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:18.843596 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:18.843596 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:18.843643 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:18.843902 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:18.843902 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:18.843939 (+ 37us) client_negotiation.cc:769] Sending connection context
0924 21:12:18.844002 (+ 63us) client_negotiation.cc:240] Negotiation successful
0924 21:12:18.844104 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":64,"threads_started":1}
I20260924 21:12:18.848129 6986 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 24 21:12:18 dist-test-slave-q276 krb5kdc[8056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284338, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 21:12:18 dist-test-slave-q276 krb5kdc[8056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284338, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:18.870183 8154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:18.863737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34436 (local address 127.6.210.190:39053)
0924 21:12:18.863799 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:18.863801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:18.863809 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:12:18.863833 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:18.863834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:18.863842 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:18.863892 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:18.864426 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.864904 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.865610 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.865777 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.865825 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:18.867958 (+ 2133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:18.867960 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:18.867960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:18.867968 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:18.869311 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:18.869592 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:18.869593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:18.869593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:18.869640 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:18.869779 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:18.869780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:18.869780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:18.869866 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:18.869933 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:18.870039 (+ 106us) server_negotiation.cc:299] Negotiation successful
0924 21:12:18.870094 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 21:12:18.871758 8100 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34436:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:12:18.873242 8100 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:12:18.879691 8601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:18.875471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:40047 (local address 127.0.0.1:38534)
0924 21:12:18.875764 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:18.875804 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:18.875872 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:18.876136 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:18.876139 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:18.876140 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:18.876199 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:18.876200 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.877388 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.877389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:18.878688 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:18.878692 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.879360 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.879361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:18.879444 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:18.879457 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 21:12:18.879551 (+ 94us) client_negotiation.cc:240] Negotiation successful
0924 21:12:18.879603 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":72,"threads_started":1}
I20260924 21:12:18.879825 8606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:18.875953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45384 (local address 127.6.210.129:42409)
0924 21:12:18.876150 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:18.876155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:18.876164 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:12:18.876203 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:18.876205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:18.876220 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:18.876423 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:18.876523 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.877625 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.878712 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.879446 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.879534 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:18.879590 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:18.879667 (+ 77us) server_negotiation.cc:299] Negotiation successful
0924 21:12:18.879702 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":89,"threads_started":1}
I20260924 21:12:18.879920 8603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:18.875681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:34611 (local address 127.0.0.1:51146)
0924 21:12:18.875955 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:18.875964 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:18.876005 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:18.876309 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:18.876309 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:18.876311 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:18.876338 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:18.876339 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.877609 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.877610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:18.878710 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:18.878712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.879718 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.879718 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:18.879775 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:18.879779 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:12:18.879829 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 21:12:18.879866 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":63,"threads_started":1}
I20260924 21:12:18.880018 8605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:18.875839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:42409 (local address 127.0.0.1:45384)
0924 21:12:18.875978 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:18.875989 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:18.876045 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:18.876414 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:18.876415 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:18.876416 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:18.876450 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:18.876451 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.877701 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.877702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:18.878633 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:18.878635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.879541 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.879542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:18.879593 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:18.879597 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:12:18.879654 (+ 57us) client_negotiation.cc:240] Negotiation successful
0924 21:12:18.879683 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
I20260924 21:12:18.880270 8604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:18.875787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51146 (local address 127.6.210.130:34611)
0924 21:12:18.876153 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:18.876157 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:18.876165 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:12:18.876203 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:18.876205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:18.876220 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:18.876333 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:18.876395 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.877514 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.878777 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.879462 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.879534 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:18.879590 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:18.880130 (+ 540us) server_negotiation.cc:299] Negotiation successful
0924 21:12:18.880176 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":95,"threads_started":1}
I20260924 21:12:18.880300 8602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:18.875573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38534 (local address 127.6.210.131:40047)
0924 21:12:18.875729 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:18.875735 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:18.875992 (+ 257us) server_negotiation.cc:407] Connection header received
0924 21:12:18.876038 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:18.876041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:18.876058 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:18.876126 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:18.876252 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.877305 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.878789 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.879273 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.879361 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:18.879422 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:18.880164 (+ 742us) server_negotiation.cc:299] Negotiation successful
0924 21:12:18.880202 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
I20260924 21:12:18.880720 8238 tablet_service.cc:1467] Processing CreateTablet for tablet 9ad322d53be446ad8f5d10761286ec6c (DEFAULT_TABLE table=test-table [id=a1bea07761464dcdbc04e8c36fbd174c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:18.880872 8518 tablet_service.cc:1467] Processing CreateTablet for tablet 9ad322d53be446ad8f5d10761286ec6c (DEFAULT_TABLE table=test-table [id=a1bea07761464dcdbc04e8c36fbd174c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:18.880872 8378 tablet_service.cc:1467] Processing CreateTablet for tablet 9ad322d53be446ad8f5d10761286ec6c (DEFAULT_TABLE table=test-table [id=a1bea07761464dcdbc04e8c36fbd174c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:18.881091 8238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ad322d53be446ad8f5d10761286ec6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:18.881189 8518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ad322d53be446ad8f5d10761286ec6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:18.881189 8378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ad322d53be446ad8f5d10761286ec6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:18.884066 8607 tablet_bootstrap.cc:492] T 9ad322d53be446ad8f5d10761286ec6c P 205cc24777c74b22b462a666e3216372: Bootstrap starting.
I20260924 21:12:18.884323 8608 tablet_bootstrap.cc:492] T 9ad322d53be446ad8f5d10761286ec6c P e886851288674ad4bf7ba99c9d769e3d: Bootstrap starting.
I20260924 21:12:18.884555 8609 tablet_bootstrap.cc:492] T 9ad322d53be446ad8f5d10761286ec6c P 82d3aabdbec945738a0b35a8b2db05ec: Bootstrap starting.
I20260924 21:12:18.884740 8607 tablet_bootstrap.cc:654] T 9ad322d53be446ad8f5d10761286ec6c P 205cc24777c74b22b462a666e3216372: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:18.885150 8609 tablet_bootstrap.cc:654] T 9ad322d53be446ad8f5d10761286ec6c P 82d3aabdbec945738a0b35a8b2db05ec: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:18.885151 8607 log.cc:824] T 9ad322d53be446ad8f5d10761286ec6c P 205cc24777c74b22b462a666e3216372: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:18.885387 8608 tablet_bootstrap.cc:654] T 9ad322d53be446ad8f5d10761286ec6c P e886851288674ad4bf7ba99c9d769e3d: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:18.885486 8609 log.cc:824] T 9ad322d53be446ad8f5d10761286ec6c P 82d3aabdbec945738a0b35a8b2db05ec: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:18.885739 8608 log.cc:824] T 9ad322d53be446ad8f5d10761286ec6c P e886851288674ad4bf7ba99c9d769e3d: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:18.886241 8609 tablet_bootstrap.cc:492] T 9ad322d53be446ad8f5d10761286ec6c P 82d3aabdbec945738a0b35a8b2db05ec: No bootstrap required, opened a new log
I20260924 21:12:18.886330 8609 ts_tablet_manager.cc:1397] T 9ad322d53be446ad8f5d10761286ec6c P 82d3aabdbec945738a0b35a8b2db05ec: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:12:18.886437 8608 tablet_bootstrap.cc:492] T 9ad322d53be446ad8f5d10761286ec6c P e886851288674ad4bf7ba99c9d769e3d: No bootstrap required, opened a new log
I20260924 21:12:18.886507 8608 ts_tablet_manager.cc:1397] T 9ad322d53be446ad8f5d10761286ec6c P e886851288674ad4bf7ba99c9d769e3d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:12:18.888263 8609 raft_consensus.cc:348] T 9ad322d53be446ad8f5d10761286ec6c P 82d3aabdbec945738a0b35a8b2db05ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } }
I20260924 21:12:18.888325 8608 raft_consensus.cc:348] T 9ad322d53be446ad8f5d10761286ec6c P e886851288674ad4bf7ba99c9d769e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } }
I20260924 21:12:18.888432 8609 raft_consensus.cc:374] T 9ad322d53be446ad8f5d10761286ec6c P 82d3aabdbec945738a0b35a8b2db05ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:18.888441 8608 raft_consensus.cc:374] T 9ad322d53be446ad8f5d10761286ec6c P e886851288674ad4bf7ba99c9d769e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:18.888485 8608 raft_consensus.cc:729] T 9ad322d53be446ad8f5d10761286ec6c P e886851288674ad4bf7ba99c9d769e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e886851288674ad4bf7ba99c9d769e3d, State: Initialized, Role: FOLLOWER
I20260924 21:12:18.888473 8609 raft_consensus.cc:729] T 9ad322d53be446ad8f5d10761286ec6c P 82d3aabdbec945738a0b35a8b2db05ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82d3aabdbec945738a0b35a8b2db05ec, State: Initialized, Role: FOLLOWER
I20260924 21:12:18.888602 8609 consensus_queue.cc:260] T 9ad322d53be446ad8f5d10761286ec6c P 82d3aabdbec945738a0b35a8b2db05ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } }
I20260924 21:12:18.888602 8608 consensus_queue.cc:260] T 9ad322d53be446ad8f5d10761286ec6c P e886851288674ad4bf7ba99c9d769e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } }
I20260924 21:12:18.888854 8609 ts_tablet_manager.cc:1428] T 9ad322d53be446ad8f5d10761286ec6c P 82d3aabdbec945738a0b35a8b2db05ec: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260924 21:12:18.888854 8608 ts_tablet_manager.cc:1428] T 9ad322d53be446ad8f5d10761286ec6c P e886851288674ad4bf7ba99c9d769e3d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:12:18.888880 8584 heartbeater.cc:503] Master 127.6.210.190:39053 was elected leader, sending a full tablet report...
I20260924 21:12:18.888969 8444 heartbeater.cc:503] Master 127.6.210.190:39053 was elected leader, sending a full tablet report...
I20260924 21:12:18.889362 8607 tablet_bootstrap.cc:492] T 9ad322d53be446ad8f5d10761286ec6c P 205cc24777c74b22b462a666e3216372: No bootstrap required, opened a new log
I20260924 21:12:18.889431 8607 ts_tablet_manager.cc:1397] T 9ad322d53be446ad8f5d10761286ec6c P 205cc24777c74b22b462a666e3216372: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260924 21:12:18.891065 8607 raft_consensus.cc:348] T 9ad322d53be446ad8f5d10761286ec6c P 205cc24777c74b22b462a666e3216372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } }
I20260924 21:12:18.891183 8607 raft_consensus.cc:374] T 9ad322d53be446ad8f5d10761286ec6c P 205cc24777c74b22b462a666e3216372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:18.891204 8607 raft_consensus.cc:729] T 9ad322d53be446ad8f5d10761286ec6c P 205cc24777c74b22b462a666e3216372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 205cc24777c74b22b462a666e3216372, State: Initialized, Role: FOLLOWER
I20260924 21:12:18.891283 8607 consensus_queue.cc:260] T 9ad322d53be446ad8f5d10761286ec6c P 205cc24777c74b22b462a666e3216372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } }
I20260924 21:12:18.891459 8607 ts_tablet_manager.cc:1428] T 9ad322d53be446ad8f5d10761286ec6c P 205cc24777c74b22b462a666e3216372: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:12:18.891541 8304 heartbeater.cc:503] Master 127.6.210.190:39053 was elected leader, sending a full tablet report...
W20260924 21:12:18.921623 8445 tablet.cc:2369] T 9ad322d53be446ad8f5d10761286ec6c P 82d3aabdbec945738a0b35a8b2db05ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:12:18.940346 8613 raft_consensus.cc:482] T 9ad322d53be446ad8f5d10761286ec6c P e886851288674ad4bf7ba99c9d769e3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:12:18.940469 8613 raft_consensus.cc:504] T 9ad322d53be446ad8f5d10761286ec6c P e886851288674ad4bf7ba99c9d769e3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } }
I20260924 21:12:18.940829 8613 leader_election.cc:288] T 9ad322d53be446ad8f5d10761286ec6c P e886851288674ad4bf7ba99c9d769e3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 82d3aabdbec945738a0b35a8b2db05ec (127.6.210.130:34611), 205cc24777c74b22b462a666e3216372 (127.6.210.129:42409)
I20260924 21:12:18.943831 8477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:18.941051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:34611 (local address 127.6.210.131:34949)
0924 21:12:18.941115 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:18.941128 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:18.941171 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:18.941333 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:18.941334 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:18.941336 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:18.941401 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:18.941403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.942076 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.942077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:18.942965 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:18.942967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.943570 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.943571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:18.943661 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:18.943664 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:12:18.943720 (+ 56us) client_negotiation.cc:240] Negotiation successful
0924 21:12:18.943759 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260924 21:12:18.943856 8604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:18.941134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:34949 (local address 127.6.210.130:34611)
0924 21:12:18.941202 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:18.941204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:18.941219 (+ 15us) server_negotiation.cc:407] Connection header received
0924 21:12:18.941244 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:18.941244 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:18.941253 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:18.941328 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:18.941485 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.941998 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.943048 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.943484 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.943562 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:18.943571 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:18.943739 (+ 168us) server_negotiation.cc:299] Negotiation successful
0924 21:12:18.943776 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 21:12:18.944095 8606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:18.941337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:44513 (local address 127.6.210.129:42409)
0924 21:12:18.941516 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:18.941518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:18.941524 (+ 6us) server_negotiation.cc:407] Connection header received
0924 21:12:18.941544 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:18.941545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:18.941552 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:18.941621 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:18.941788 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.942336 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.943269 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.943656 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.943740 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:18.943747 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:18.943981 (+ 234us) server_negotiation.cc:299] Negotiation successful
0924 21:12:18.944033 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159}
I20260924 21:12:18.944168 8398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ad322d53be446ad8f5d10761286ec6c" candidate_uuid: "e886851288674ad4bf7ba99c9d769e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82d3aabdbec945738a0b35a8b2db05ec" is_pre_election: true
I20260924 21:12:18.944337 8398 raft_consensus.cc:2454] T 9ad322d53be446ad8f5d10761286ec6c P 82d3aabdbec945738a0b35a8b2db05ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e886851288674ad4bf7ba99c9d769e3d in term 0.
I20260924 21:12:18.944353 8616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:18.941191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:42409 (local address 127.6.210.131:44513)
0924 21:12:18.941364 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:18.941373 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:18.941431 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:18.941627 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:18.941628 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:18.941629 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:18.941709 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:18.941710 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.942418 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.942419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:18.943185 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:18.943187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.943750 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.943752 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:18.943848 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:18.943852 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:12:18.944222 (+ 370us) client_negotiation.cc:240] Negotiation successful
0924 21:12:18.944264 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":101,"threads_started":1}
I20260924 21:12:18.944602 8471 leader_election.cc:302] T 9ad322d53be446ad8f5d10761286ec6c P e886851288674ad4bf7ba99c9d769e3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82d3aabdbec945738a0b35a8b2db05ec, e886851288674ad4bf7ba99c9d769e3d; no voters:
I20260924 21:12:18.944672 8258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ad322d53be446ad8f5d10761286ec6c" candidate_uuid: "e886851288674ad4bf7ba99c9d769e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "205cc24777c74b22b462a666e3216372" is_pre_election: true
I20260924 21:12:18.944761 8613 raft_consensus.cc:2781] T 9ad322d53be446ad8f5d10761286ec6c P e886851288674ad4bf7ba99c9d769e3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:12:18.944825 8613 raft_consensus.cc:482] T 9ad322d53be446ad8f5d10761286ec6c P e886851288674ad4bf7ba99c9d769e3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:12:18.944830 8258 raft_consensus.cc:2454] T 9ad322d53be446ad8f5d10761286ec6c P 205cc24777c74b22b462a666e3216372 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e886851288674ad4bf7ba99c9d769e3d in term 0.
I20260924 21:12:18.944861 8613 raft_consensus.cc:3037] T 9ad322d53be446ad8f5d10761286ec6c P e886851288674ad4bf7ba99c9d769e3d [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:18.945667 8613 raft_consensus.cc:504] T 9ad322d53be446ad8f5d10761286ec6c P e886851288674ad4bf7ba99c9d769e3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } }
I20260924 21:12:18.945784 8613 leader_election.cc:288] T 9ad322d53be446ad8f5d10761286ec6c P e886851288674ad4bf7ba99c9d769e3d [CANDIDATE]: Term 1 election: Requested vote from peers 82d3aabdbec945738a0b35a8b2db05ec (127.6.210.130:34611), 205cc24777c74b22b462a666e3216372 (127.6.210.129:42409)
I20260924 21:12:18.946026 8258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ad322d53be446ad8f5d10761286ec6c" candidate_uuid: "e886851288674ad4bf7ba99c9d769e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "205cc24777c74b22b462a666e3216372"
I20260924 21:12:18.946043 8398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ad322d53be446ad8f5d10761286ec6c" candidate_uuid: "e886851288674ad4bf7ba99c9d769e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82d3aabdbec945738a0b35a8b2db05ec"
I20260924 21:12:18.946121 8398 raft_consensus.cc:3037] T 9ad322d53be446ad8f5d10761286ec6c P 82d3aabdbec945738a0b35a8b2db05ec [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:18.946126 8258 raft_consensus.cc:3037] T 9ad322d53be446ad8f5d10761286ec6c P 205cc24777c74b22b462a666e3216372 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:18.946867 8398 raft_consensus.cc:2454] T 9ad322d53be446ad8f5d10761286ec6c P 82d3aabdbec945738a0b35a8b2db05ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e886851288674ad4bf7ba99c9d769e3d in term 1.
I20260924 21:12:18.946970 8258 raft_consensus.cc:2454] T 9ad322d53be446ad8f5d10761286ec6c P 205cc24777c74b22b462a666e3216372 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e886851288674ad4bf7ba99c9d769e3d in term 1.
I20260924 21:12:18.947028 8471 leader_election.cc:302] T 9ad322d53be446ad8f5d10761286ec6c P e886851288674ad4bf7ba99c9d769e3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82d3aabdbec945738a0b35a8b2db05ec, e886851288674ad4bf7ba99c9d769e3d; no voters:
I20260924 21:12:18.947130 8613 raft_consensus.cc:2781] T 9ad322d53be446ad8f5d10761286ec6c P e886851288674ad4bf7ba99c9d769e3d [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:18.947317 8613 raft_consensus.cc:686] T 9ad322d53be446ad8f5d10761286ec6c P e886851288674ad4bf7ba99c9d769e3d [term 1 LEADER]: Becoming Leader. State: Replica: e886851288674ad4bf7ba99c9d769e3d, State: Running, Role: LEADER
I20260924 21:12:18.947419 8613 consensus_queue.cc:237] T 9ad322d53be446ad8f5d10761286ec6c P e886851288674ad4bf7ba99c9d769e3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } }
I20260924 21:12:18.948333 8101 catalog_manager.cc:5495] T 9ad322d53be446ad8f5d10761286ec6c P e886851288674ad4bf7ba99c9d769e3d reported cstate change: term changed from 0 to 1, leader changed from <none> to e886851288674ad4bf7ba99c9d769e3d (127.6.210.131). New cstate: current_term: 1 leader_uuid: "e886851288674ad4bf7ba99c9d769e3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } health_report { overall_health: UNKNOWN } } }
I20260924 21:12:18.954175 8602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:18.951789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38548 (local address 127.6.210.131:40047)
0924 21:12:18.951828 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:18.951831 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:18.951840 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:12:18.951901 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:18.951901 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:18.951914 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:18.951972 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:12:18.952094 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.952525 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.953226 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:18.953393 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:18.953440 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:18.953643 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:12:18.953756 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:12:18.953845 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:18.953885 (+ 40us) server_negotiation.cc:299] Negotiation successful
0924 21:12:18.953902 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 21:12:18.956117 8258 raft_consensus.cc:1261] T 9ad322d53be446ad8f5d10761286ec6c P 205cc24777c74b22b462a666e3216372 [term 1 FOLLOWER]: Refusing update from remote peer e886851288674ad4bf7ba99c9d769e3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:18.956117 8398 raft_consensus.cc:1261] T 9ad322d53be446ad8f5d10761286ec6c P 82d3aabdbec945738a0b35a8b2db05ec [term 1 FOLLOWER]: Refusing update from remote peer e886851288674ad4bf7ba99c9d769e3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:18.956432 8617 consensus_queue.cc:1035] T 9ad322d53be446ad8f5d10761286ec6c P e886851288674ad4bf7ba99c9d769e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:12:18.956568 8613 consensus_queue.cc:1035] T 9ad322d53be446ad8f5d10761286ec6c P e886851288674ad4bf7ba99c9d769e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:12:18.958751 8620 mvcc.cc:205] Tried to move back new op lower bound from 7333004652361953280 to 7333004652329512960. Current Snapshot: MvccSnapshot[applied={T|T < 7333004652361953280}]
I20260924 21:12:18.959323 8622 mvcc.cc:205] Tried to move back new op lower bound from 7333004652361953280 to 7333004652329512960. Current Snapshot: MvccSnapshot[applied={T|T < 7333004652361953280}]
I20260924 21:12:18.962291 8101 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:12:18.962393 8101 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:12:18.963109 8101 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 271d3ea1dc374c169abd29787d688cb9: Sending DeleteTablet for 3 replicas of tablet 9ad322d53be446ad8f5d10761286ec6c
I20260924 21:12:18.963444 8518 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ad322d53be446ad8f5d10761286ec6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:18 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:38534
I20260924 21:12:18.963479 8238 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ad322d53be446ad8f5d10761286ec6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:18 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:45384
I20260924 21:12:18.963658 8629 tablet_replica.cc:330] T 9ad322d53be446ad8f5d10761286ec6c P 205cc24777c74b22b462a666e3216372: stopping tablet replica
I20260924 21:12:18.963706 8378 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ad322d53be446ad8f5d10761286ec6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:18 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:51146
I20260924 21:12:18.963740 8629 raft_consensus.cc:2229] T 9ad322d53be446ad8f5d10761286ec6c P 205cc24777c74b22b462a666e3216372 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:12:18.963680 8101 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34436:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:12:18.963804 8101 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:12:18.963845 8629 raft_consensus.cc:2258] T 9ad322d53be446ad8f5d10761286ec6c P 205cc24777c74b22b462a666e3216372 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:18.964228 8630 tablet_replica.cc:330] T 9ad322d53be446ad8f5d10761286ec6c P e886851288674ad4bf7ba99c9d769e3d: stopping tablet replica
I20260924 21:12:18.964293 8629 ts_tablet_manager.cc:1905] T 9ad322d53be446ad8f5d10761286ec6c P 205cc24777c74b22b462a666e3216372: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:18.964406 8631 tablet_replica.cc:330] T 9ad322d53be446ad8f5d10761286ec6c P 82d3aabdbec945738a0b35a8b2db05ec: stopping tablet replica
I20260924 21:12:18.964538 8631 raft_consensus.cc:2229] T 9ad322d53be446ad8f5d10761286ec6c P 82d3aabdbec945738a0b35a8b2db05ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:12:18.964689 8630 raft_consensus.cc:2229] T 9ad322d53be446ad8f5d10761286ec6c P e886851288674ad4bf7ba99c9d769e3d [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:12:18.964923 8238 tablet_service.cc:1467] Processing CreateTablet for tablet 16c2066b14ec4fbc8d18af768763a9cb (DEFAULT_TABLE table=test-table [id=1f89b4ffc1b047819dce8941d977e6a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:18.965035 8238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16c2066b14ec4fbc8d18af768763a9cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:18.965247 8378 tablet_service.cc:1467] Processing CreateTablet for tablet 16c2066b14ec4fbc8d18af768763a9cb (DEFAULT_TABLE table=test-table [id=1f89b4ffc1b047819dce8941d977e6a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:18.965332 8378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16c2066b14ec4fbc8d18af768763a9cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:18.966354 8607 tablet_bootstrap.cc:492] T 16c2066b14ec4fbc8d18af768763a9cb P 205cc24777c74b22b462a666e3216372: Bootstrap starting.
I20260924 21:12:18.966701 8609 tablet_bootstrap.cc:492] T 16c2066b14ec4fbc8d18af768763a9cb P 82d3aabdbec945738a0b35a8b2db05ec: Bootstrap starting.
I20260924 21:12:18.966795 8607 tablet_bootstrap.cc:654] T 16c2066b14ec4fbc8d18af768763a9cb P 205cc24777c74b22b462a666e3216372: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:18.967092 8609 tablet_bootstrap.cc:654] T 16c2066b14ec4fbc8d18af768763a9cb P 82d3aabdbec945738a0b35a8b2db05ec: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:18.964907 8518 tablet_service.cc:1467] Processing CreateTablet for tablet 16c2066b14ec4fbc8d18af768763a9cb (DEFAULT_TABLE table=test-table [id=1f89b4ffc1b047819dce8941d977e6a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:18.967590 8518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16c2066b14ec4fbc8d18af768763a9cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:18.967729 8607 tablet_bootstrap.cc:492] T 16c2066b14ec4fbc8d18af768763a9cb P 205cc24777c74b22b462a666e3216372: No bootstrap required, opened a new log
I20260924 21:12:18.967769 8607 ts_tablet_manager.cc:1397] T 16c2066b14ec4fbc8d18af768763a9cb P 205cc24777c74b22b462a666e3216372: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:18.967903 8607 raft_consensus.cc:348] T 16c2066b14ec4fbc8d18af768763a9cb P 205cc24777c74b22b462a666e3216372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } }
I20260924 21:12:18.967965 8607 raft_consensus.cc:374] T 16c2066b14ec4fbc8d18af768763a9cb P 205cc24777c74b22b462a666e3216372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:18.967986 8607 raft_consensus.cc:729] T 16c2066b14ec4fbc8d18af768763a9cb P 205cc24777c74b22b462a666e3216372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 205cc24777c74b22b462a666e3216372, State: Initialized, Role: FOLLOWER
I20260924 21:12:18.968011 8629 ts_tablet_manager.cc:1918] T 9ad322d53be446ad8f5d10761286ec6c P 205cc24777c74b22b462a666e3216372: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:12:18.968029 8607 consensus_queue.cc:260] T 16c2066b14ec4fbc8d18af768763a9cb P 205cc24777c74b22b462a666e3216372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } }
I20260924 21:12:18.968070 8629 log.cc:1196] T 9ad322d53be446ad8f5d10761286ec6c P 205cc24777c74b22b462a666e3216372: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-0/wal/wals/9ad322d53be446ad8f5d10761286ec6c
I20260924 21:12:18.968113 8607 ts_tablet_manager.cc:1428] T 16c2066b14ec4fbc8d18af768763a9cb P 205cc24777c74b22b462a666e3216372: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:18.968328 8629 ts_tablet_manager.cc:1939] T 9ad322d53be446ad8f5d10761286ec6c P 205cc24777c74b22b462a666e3216372: Deleting consensus metadata
I20260924 21:12:18.968689 8087 catalog_manager.cc:4841] TS 205cc24777c74b22b462a666e3216372 (127.6.210.129:42409): tablet 9ad322d53be446ad8f5d10761286ec6c (table test-table [id=a1bea07761464dcdbc04e8c36fbd174c]) successfully deleted
I20260924 21:12:18.968873 8609 tablet_bootstrap.cc:492] T 16c2066b14ec4fbc8d18af768763a9cb P 82d3aabdbec945738a0b35a8b2db05ec: No bootstrap required, opened a new log
I20260924 21:12:18.968916 8609 ts_tablet_manager.cc:1397] T 16c2066b14ec4fbc8d18af768763a9cb P 82d3aabdbec945738a0b35a8b2db05ec: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 21:12:18.969004 8608 tablet_bootstrap.cc:492] T 16c2066b14ec4fbc8d18af768763a9cb P e886851288674ad4bf7ba99c9d769e3d: Bootstrap starting.
I20260924 21:12:18.969060 8609 raft_consensus.cc:348] T 16c2066b14ec4fbc8d18af768763a9cb P 82d3aabdbec945738a0b35a8b2db05ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } }
I20260924 21:12:18.969126 8609 raft_consensus.cc:374] T 16c2066b14ec4fbc8d18af768763a9cb P 82d3aabdbec945738a0b35a8b2db05ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:18.969147 8609 raft_consensus.cc:729] T 16c2066b14ec4fbc8d18af768763a9cb P 82d3aabdbec945738a0b35a8b2db05ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82d3aabdbec945738a0b35a8b2db05ec, State: Initialized, Role: FOLLOWER
I20260924 21:12:18.969190 8609 consensus_queue.cc:260] T 16c2066b14ec4fbc8d18af768763a9cb P 82d3aabdbec945738a0b35a8b2db05ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } }
I20260924 21:12:18.969275 8609 ts_tablet_manager.cc:1428] T 16c2066b14ec4fbc8d18af768763a9cb P 82d3aabdbec945738a0b35a8b2db05ec: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:18.969518 8608 tablet_bootstrap.cc:654] T 16c2066b14ec4fbc8d18af768763a9cb P e886851288674ad4bf7ba99c9d769e3d: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:18.970201 8608 tablet_bootstrap.cc:492] T 16c2066b14ec4fbc8d18af768763a9cb P e886851288674ad4bf7ba99c9d769e3d: No bootstrap required, opened a new log
I20260924 21:12:18.970256 8608 ts_tablet_manager.cc:1397] T 16c2066b14ec4fbc8d18af768763a9cb P e886851288674ad4bf7ba99c9d769e3d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:18.970363 8630 raft_consensus.cc:2258] T 9ad322d53be446ad8f5d10761286ec6c P e886851288674ad4bf7ba99c9d769e3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:18.970432 8608 raft_consensus.cc:348] T 16c2066b14ec4fbc8d18af768763a9cb P e886851288674ad4bf7ba99c9d769e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } }
I20260924 21:12:18.970496 8631 raft_consensus.cc:2258] T 9ad322d53be446ad8f5d10761286ec6c P 82d3aabdbec945738a0b35a8b2db05ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:18.970511 8608 raft_consensus.cc:374] T 16c2066b14ec4fbc8d18af768763a9cb P e886851288674ad4bf7ba99c9d769e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:18.970537 8608 raft_consensus.cc:729] T 16c2066b14ec4fbc8d18af768763a9cb P e886851288674ad4bf7ba99c9d769e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e886851288674ad4bf7ba99c9d769e3d, State: Initialized, Role: FOLLOWER
I20260924 21:12:18.970577 8608 consensus_queue.cc:260] T 16c2066b14ec4fbc8d18af768763a9cb P e886851288674ad4bf7ba99c9d769e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } }
I20260924 21:12:18.970667 8608 ts_tablet_manager.cc:1428] T 16c2066b14ec4fbc8d18af768763a9cb P e886851288674ad4bf7ba99c9d769e3d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:18.970973 8631 ts_tablet_manager.cc:1905] T 9ad322d53be446ad8f5d10761286ec6c P 82d3aabdbec945738a0b35a8b2db05ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:18.972280 8630 ts_tablet_manager.cc:1905] T 9ad322d53be446ad8f5d10761286ec6c P e886851288674ad4bf7ba99c9d769e3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:18.972337 8631 ts_tablet_manager.cc:1918] T 9ad322d53be446ad8f5d10761286ec6c P 82d3aabdbec945738a0b35a8b2db05ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:12:18.972389 8631 log.cc:1196] T 9ad322d53be446ad8f5d10761286ec6c P 82d3aabdbec945738a0b35a8b2db05ec: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-1/wal/wals/9ad322d53be446ad8f5d10761286ec6c
I20260924 21:12:18.972646 8631 ts_tablet_manager.cc:1939] T 9ad322d53be446ad8f5d10761286ec6c P 82d3aabdbec945738a0b35a8b2db05ec: Deleting consensus metadata
I20260924 21:12:18.973057 8087 catalog_manager.cc:4841] TS 82d3aabdbec945738a0b35a8b2db05ec (127.6.210.130:34611): tablet 9ad322d53be446ad8f5d10761286ec6c (table test-table [id=a1bea07761464dcdbc04e8c36fbd174c]) successfully deleted
I20260924 21:12:18.973663 8630 ts_tablet_manager.cc:1918] T 9ad322d53be446ad8f5d10761286ec6c P e886851288674ad4bf7ba99c9d769e3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:12:18.973717 8630 log.cc:1196] T 9ad322d53be446ad8f5d10761286ec6c P e886851288674ad4bf7ba99c9d769e3d: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-2/wal/wals/9ad322d53be446ad8f5d10761286ec6c
I20260924 21:12:18.973955 8630 ts_tablet_manager.cc:1939] T 9ad322d53be446ad8f5d10761286ec6c P e886851288674ad4bf7ba99c9d769e3d: Deleting consensus metadata
I20260924 21:12:18.974318 8087 catalog_manager.cc:4841] TS e886851288674ad4bf7ba99c9d769e3d (127.6.210.131:40047): tablet 9ad322d53be446ad8f5d10761286ec6c (table test-table [id=a1bea07761464dcdbc04e8c36fbd174c]) successfully deleted
W20260924 21:12:19.013618 8305 tablet.cc:2369] T 16c2066b14ec4fbc8d18af768763a9cb P 205cc24777c74b22b462a666e3216372: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:12:19.084290 8613 raft_consensus.cc:482] T 16c2066b14ec4fbc8d18af768763a9cb P e886851288674ad4bf7ba99c9d769e3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:12:19.084384 8613 raft_consensus.cc:504] T 16c2066b14ec4fbc8d18af768763a9cb P e886851288674ad4bf7ba99c9d769e3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } }
I20260924 21:12:19.084546 8613 leader_election.cc:288] T 16c2066b14ec4fbc8d18af768763a9cb P e886851288674ad4bf7ba99c9d769e3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 205cc24777c74b22b462a666e3216372 (127.6.210.129:42409), 82d3aabdbec945738a0b35a8b2db05ec (127.6.210.130:34611)
I20260924 21:12:19.084771 8258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16c2066b14ec4fbc8d18af768763a9cb" candidate_uuid: "e886851288674ad4bf7ba99c9d769e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "205cc24777c74b22b462a666e3216372" is_pre_election: true
I20260924 21:12:19.084784 8398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16c2066b14ec4fbc8d18af768763a9cb" candidate_uuid: "e886851288674ad4bf7ba99c9d769e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82d3aabdbec945738a0b35a8b2db05ec" is_pre_election: true
I20260924 21:12:19.084857 8258 raft_consensus.cc:2454] T 16c2066b14ec4fbc8d18af768763a9cb P 205cc24777c74b22b462a666e3216372 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e886851288674ad4bf7ba99c9d769e3d in term 0.
I20260924 21:12:19.084919 8398 raft_consensus.cc:2454] T 16c2066b14ec4fbc8d18af768763a9cb P 82d3aabdbec945738a0b35a8b2db05ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e886851288674ad4bf7ba99c9d769e3d in term 0.
W20260924 21:12:19.084971 8585 tablet.cc:2369] T 16c2066b14ec4fbc8d18af768763a9cb P e886851288674ad4bf7ba99c9d769e3d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:12:19.085093 8471 leader_election.cc:302] T 16c2066b14ec4fbc8d18af768763a9cb P e886851288674ad4bf7ba99c9d769e3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82d3aabdbec945738a0b35a8b2db05ec, e886851288674ad4bf7ba99c9d769e3d; no voters:
I20260924 21:12:19.085223 8613 raft_consensus.cc:2781] T 16c2066b14ec4fbc8d18af768763a9cb P e886851288674ad4bf7ba99c9d769e3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:12:19.085270 8613 raft_consensus.cc:482] T 16c2066b14ec4fbc8d18af768763a9cb P e886851288674ad4bf7ba99c9d769e3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:12:19.085294 8613 raft_consensus.cc:3037] T 16c2066b14ec4fbc8d18af768763a9cb P e886851288674ad4bf7ba99c9d769e3d [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:19.085920 8613 raft_consensus.cc:504] T 16c2066b14ec4fbc8d18af768763a9cb P e886851288674ad4bf7ba99c9d769e3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } }
I20260924 21:12:19.086092 8613 leader_election.cc:288] T 16c2066b14ec4fbc8d18af768763a9cb P e886851288674ad4bf7ba99c9d769e3d [CANDIDATE]: Term 1 election: Requested vote from peers 205cc24777c74b22b462a666e3216372 (127.6.210.129:42409), 82d3aabdbec945738a0b35a8b2db05ec (127.6.210.130:34611)
I20260924 21:12:19.086303 8258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16c2066b14ec4fbc8d18af768763a9cb" candidate_uuid: "e886851288674ad4bf7ba99c9d769e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "205cc24777c74b22b462a666e3216372"
I20260924 21:12:19.086331 8398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16c2066b14ec4fbc8d18af768763a9cb" candidate_uuid: "e886851288674ad4bf7ba99c9d769e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82d3aabdbec945738a0b35a8b2db05ec"
I20260924 21:12:19.086390 8398 raft_consensus.cc:3037] T 16c2066b14ec4fbc8d18af768763a9cb P 82d3aabdbec945738a0b35a8b2db05ec [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:19.086390 8258 raft_consensus.cc:3037] T 16c2066b14ec4fbc8d18af768763a9cb P 205cc24777c74b22b462a666e3216372 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:19.087138 8258 raft_consensus.cc:2454] T 16c2066b14ec4fbc8d18af768763a9cb P 205cc24777c74b22b462a666e3216372 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e886851288674ad4bf7ba99c9d769e3d in term 1.
I20260924 21:12:19.087138 8398 raft_consensus.cc:2454] T 16c2066b14ec4fbc8d18af768763a9cb P 82d3aabdbec945738a0b35a8b2db05ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e886851288674ad4bf7ba99c9d769e3d in term 1.
I20260924 21:12:19.087303 8471 leader_election.cc:302] T 16c2066b14ec4fbc8d18af768763a9cb P e886851288674ad4bf7ba99c9d769e3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 205cc24777c74b22b462a666e3216372, e886851288674ad4bf7ba99c9d769e3d; no voters:
I20260924 21:12:19.087391 8613 raft_consensus.cc:2781] T 16c2066b14ec4fbc8d18af768763a9cb P e886851288674ad4bf7ba99c9d769e3d [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:19.087451 8613 raft_consensus.cc:686] T 16c2066b14ec4fbc8d18af768763a9cb P e886851288674ad4bf7ba99c9d769e3d [term 1 LEADER]: Becoming Leader. State: Replica: e886851288674ad4bf7ba99c9d769e3d, State: Running, Role: LEADER
I20260924 21:12:19.087500 8613 consensus_queue.cc:237] T 16c2066b14ec4fbc8d18af768763a9cb P e886851288674ad4bf7ba99c9d769e3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } }
I20260924 21:12:19.088119 8101 catalog_manager.cc:5495] T 16c2066b14ec4fbc8d18af768763a9cb P e886851288674ad4bf7ba99c9d769e3d reported cstate change: term changed from 0 to 1, leader changed from <none> to e886851288674ad4bf7ba99c9d769e3d (127.6.210.131). New cstate: current_term: 1 leader_uuid: "e886851288674ad4bf7ba99c9d769e3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } health_report { overall_health: HEALTHY } } }
Sep 24 21:12:19 dist-test-slave-q276 krb5kdc[8056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284338, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:19.146504 8154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:19.139227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34448 (local address 127.6.210.190:39053)
0924 21:12:19.139309 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:19.139333 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:19.139425 (+ 92us) server_negotiation.cc:407] Connection header received
0924 21:12:19.139504 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:19.139505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:19.139514 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:19.139587 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:19.140086 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:19.140690 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:19.141380 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:19.141569 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:19.141639 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:19.143868 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:19.143870 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:19.143870 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:19.143877 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:19.145236 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:19.145631 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:19.145632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:19.145632 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:19.145688 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:19.145893 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:19.145894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:19.145894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:19.146012 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:19.146087 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:19.146235 (+ 148us) server_negotiation.cc:299] Negotiation successful
0924 21:12:19.146367 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260924 21:12:19.146564 8641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:19.139155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39053 (local address 127.0.0.1:34448)
0924 21:12:19.139320 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:19.139350 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:19.139430 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:19.139592 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:19.139593 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:19.139946 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:19.139992 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:19.139994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:19.140764 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:19.140766 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:19.141287 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:19.141289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:19.141636 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:19.141637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:19.141685 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:19.142240 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:19.142245 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:19.143776 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:19.145329 (+ 1553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:19.145336 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:19.145339 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:19.145548 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:19.145751 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:19.145751 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:19.145752 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:19.145810 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:19.146076 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:19.146083 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:19.146128 (+ 45us) client_negotiation.cc:769] Sending connection context
0924 21:12:19.146215 (+ 87us) client_negotiation.cc:240] Negotiation successful
0924 21:12:19.146425 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":110,"thread_start_us":71,"threads_started":1}
I20260924 21:12:19.148707 8101 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:34448:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260924 21:12:19.148892 8101 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:12:19.150393 8238 tablet_service.cc:1467] Processing CreateTablet for tablet e21d0e75f05a4562a92d2615874d8dfb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0644ec92ec734a59a36fe0d5b902cad4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 21:12:19.150561 8238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e21d0e75f05a4562a92d2615874d8dfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:19.150530 8518 tablet_service.cc:1467] Processing CreateTablet for tablet e21d0e75f05a4562a92d2615874d8dfb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0644ec92ec734a59a36fe0d5b902cad4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 21:12:19.150561 8378 tablet_service.cc:1467] Processing CreateTablet for tablet e21d0e75f05a4562a92d2615874d8dfb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0644ec92ec734a59a36fe0d5b902cad4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 21:12:19.150676 8518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e21d0e75f05a4562a92d2615874d8dfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:19.150681 8378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e21d0e75f05a4562a92d2615874d8dfb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:19.151870 8608 tablet_bootstrap.cc:492] T e21d0e75f05a4562a92d2615874d8dfb P e886851288674ad4bf7ba99c9d769e3d: Bootstrap starting.
I20260924 21:12:19.152076 8609 tablet_bootstrap.cc:492] T e21d0e75f05a4562a92d2615874d8dfb P 82d3aabdbec945738a0b35a8b2db05ec: Bootstrap starting.
I20260924 21:12:19.152235 8608 tablet_bootstrap.cc:654] T e21d0e75f05a4562a92d2615874d8dfb P e886851288674ad4bf7ba99c9d769e3d: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:19.152450 8609 tablet_bootstrap.cc:654] T e21d0e75f05a4562a92d2615874d8dfb P 82d3aabdbec945738a0b35a8b2db05ec: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:19.152799 8607 tablet_bootstrap.cc:492] T e21d0e75f05a4562a92d2615874d8dfb P 205cc24777c74b22b462a666e3216372: Bootstrap starting.
I20260924 21:12:19.152860 8608 tablet_bootstrap.cc:492] T e21d0e75f05a4562a92d2615874d8dfb P e886851288674ad4bf7ba99c9d769e3d: No bootstrap required, opened a new log
I20260924 21:12:19.152913 8608 ts_tablet_manager.cc:1397] T e21d0e75f05a4562a92d2615874d8dfb P e886851288674ad4bf7ba99c9d769e3d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:19.153038 8608 raft_consensus.cc:348] T e21d0e75f05a4562a92d2615874d8dfb P e886851288674ad4bf7ba99c9d769e3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } }
I20260924 21:12:19.153134 8608 raft_consensus.cc:374] T e21d0e75f05a4562a92d2615874d8dfb P e886851288674ad4bf7ba99c9d769e3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:19.153172 8608 raft_consensus.cc:729] T e21d0e75f05a4562a92d2615874d8dfb P e886851288674ad4bf7ba99c9d769e3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e886851288674ad4bf7ba99c9d769e3d, State: Initialized, Role: FOLLOWER
I20260924 21:12:19.153223 8608 consensus_queue.cc:260] T e21d0e75f05a4562a92d2615874d8dfb P e886851288674ad4bf7ba99c9d769e3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } }
I20260924 21:12:19.153319 8608 ts_tablet_manager.cc:1428] T e21d0e75f05a4562a92d2615874d8dfb P e886851288674ad4bf7ba99c9d769e3d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:19.153333 8613 tablet_replica.cc:439] T e21d0e75f05a4562a92d2615874d8dfb P e886851288674ad4bf7ba99c9d769e3d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } } }
I20260924 21:12:19.153401 8613 tablet_replica.cc:442] T e21d0e75f05a4562a92d2615874d8dfb P e886851288674ad4bf7ba99c9d769e3d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:12:19.153616 8607 tablet_bootstrap.cc:654] T e21d0e75f05a4562a92d2615874d8dfb P 205cc24777c74b22b462a666e3216372: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:19.154126 8609 tablet_bootstrap.cc:492] T e21d0e75f05a4562a92d2615874d8dfb P 82d3aabdbec945738a0b35a8b2db05ec: No bootstrap required, opened a new log
I20260924 21:12:19.154201 8609 ts_tablet_manager.cc:1397] T e21d0e75f05a4562a92d2615874d8dfb P 82d3aabdbec945738a0b35a8b2db05ec: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 21:12:19.154254 8607 tablet_bootstrap.cc:492] T e21d0e75f05a4562a92d2615874d8dfb P 205cc24777c74b22b462a666e3216372: No bootstrap required, opened a new log
I20260924 21:12:19.154330 8607 ts_tablet_manager.cc:1397] T e21d0e75f05a4562a92d2615874d8dfb P 205cc24777c74b22b462a666e3216372: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:12:19.154366 8609 raft_consensus.cc:348] T e21d0e75f05a4562a92d2615874d8dfb P 82d3aabdbec945738a0b35a8b2db05ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } }
I20260924 21:12:19.154436 8609 raft_consensus.cc:374] T e21d0e75f05a4562a92d2615874d8dfb P 82d3aabdbec945738a0b35a8b2db05ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:19.154464 8609 raft_consensus.cc:729] T e21d0e75f05a4562a92d2615874d8dfb P 82d3aabdbec945738a0b35a8b2db05ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82d3aabdbec945738a0b35a8b2db05ec, State: Initialized, Role: FOLLOWER
I20260924 21:12:19.154503 8609 consensus_queue.cc:260] T e21d0e75f05a4562a92d2615874d8dfb P 82d3aabdbec945738a0b35a8b2db05ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } }
I20260924 21:12:19.154497 8607 raft_consensus.cc:348] T e21d0e75f05a4562a92d2615874d8dfb P 205cc24777c74b22b462a666e3216372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } }
I20260924 21:12:19.154569 8607 raft_consensus.cc:374] T e21d0e75f05a4562a92d2615874d8dfb P 205cc24777c74b22b462a666e3216372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:19.154603 8607 raft_consensus.cc:729] T e21d0e75f05a4562a92d2615874d8dfb P 205cc24777c74b22b462a666e3216372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 205cc24777c74b22b462a666e3216372, State: Initialized, Role: FOLLOWER
I20260924 21:12:19.154601 8609 ts_tablet_manager.cc:1428] T e21d0e75f05a4562a92d2615874d8dfb P 82d3aabdbec945738a0b35a8b2db05ec: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260924 21:12:19.154650 8607 consensus_queue.cc:260] T e21d0e75f05a4562a92d2615874d8dfb P 205cc24777c74b22b462a666e3216372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } }
I20260924 21:12:19.154681 8614 tablet_replica.cc:439] T e21d0e75f05a4562a92d2615874d8dfb P 82d3aabdbec945738a0b35a8b2db05ec: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } } }
I20260924 21:12:19.154745 8607 ts_tablet_manager.cc:1428] T e21d0e75f05a4562a92d2615874d8dfb P 205cc24777c74b22b462a666e3216372: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:19.154780 8614 tablet_replica.cc:442] T e21d0e75f05a4562a92d2615874d8dfb P 82d3aabdbec945738a0b35a8b2db05ec: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:12:19.154968 8615 tablet_replica.cc:439] T e21d0e75f05a4562a92d2615874d8dfb P 205cc24777c74b22b462a666e3216372: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } } }
I20260924 21:12:19.155056 8615 tablet_replica.cc:442] T e21d0e75f05a4562a92d2615874d8dfb P 205cc24777c74b22b462a666e3216372: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:12:19.297340 8613 raft_consensus.cc:482] T e21d0e75f05a4562a92d2615874d8dfb P e886851288674ad4bf7ba99c9d769e3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:12:19.297462 8613 raft_consensus.cc:504] T e21d0e75f05a4562a92d2615874d8dfb P e886851288674ad4bf7ba99c9d769e3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } }
I20260924 21:12:19.297636 8613 leader_election.cc:288] T e21d0e75f05a4562a92d2615874d8dfb P e886851288674ad4bf7ba99c9d769e3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 205cc24777c74b22b462a666e3216372 (127.6.210.129:42409), 82d3aabdbec945738a0b35a8b2db05ec (127.6.210.130:34611)
I20260924 21:12:19.297870 8258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e21d0e75f05a4562a92d2615874d8dfb" candidate_uuid: "e886851288674ad4bf7ba99c9d769e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "205cc24777c74b22b462a666e3216372" is_pre_election: true
I20260924 21:12:19.297966 8258 raft_consensus.cc:2454] T e21d0e75f05a4562a92d2615874d8dfb P 205cc24777c74b22b462a666e3216372 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e886851288674ad4bf7ba99c9d769e3d in term 0.
I20260924 21:12:19.298113 8471 leader_election.cc:302] T e21d0e75f05a4562a92d2615874d8dfb P e886851288674ad4bf7ba99c9d769e3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 205cc24777c74b22b462a666e3216372, e886851288674ad4bf7ba99c9d769e3d; no voters:
I20260924 21:12:19.298213 8613 raft_consensus.cc:2781] T e21d0e75f05a4562a92d2615874d8dfb P e886851288674ad4bf7ba99c9d769e3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:12:19.298076 8398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e21d0e75f05a4562a92d2615874d8dfb" candidate_uuid: "e886851288674ad4bf7ba99c9d769e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82d3aabdbec945738a0b35a8b2db05ec" is_pre_election: true
I20260924 21:12:19.298252 8613 raft_consensus.cc:482] T e21d0e75f05a4562a92d2615874d8dfb P e886851288674ad4bf7ba99c9d769e3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:12:19.298313 8613 raft_consensus.cc:3037] T e21d0e75f05a4562a92d2615874d8dfb P e886851288674ad4bf7ba99c9d769e3d [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:19.298321 8398 raft_consensus.cc:2454] T e21d0e75f05a4562a92d2615874d8dfb P 82d3aabdbec945738a0b35a8b2db05ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e886851288674ad4bf7ba99c9d769e3d in term 0.
I20260924 21:12:19.299153 8613 raft_consensus.cc:504] T e21d0e75f05a4562a92d2615874d8dfb P e886851288674ad4bf7ba99c9d769e3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } }
I20260924 21:12:19.299276 8613 leader_election.cc:288] T e21d0e75f05a4562a92d2615874d8dfb P e886851288674ad4bf7ba99c9d769e3d [CANDIDATE]: Term 1 election: Requested vote from peers 205cc24777c74b22b462a666e3216372 (127.6.210.129:42409), 82d3aabdbec945738a0b35a8b2db05ec (127.6.210.130:34611)
I20260924 21:12:19.299458 8258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e21d0e75f05a4562a92d2615874d8dfb" candidate_uuid: "e886851288674ad4bf7ba99c9d769e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "205cc24777c74b22b462a666e3216372"
I20260924 21:12:19.299455 8398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e21d0e75f05a4562a92d2615874d8dfb" candidate_uuid: "e886851288674ad4bf7ba99c9d769e3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82d3aabdbec945738a0b35a8b2db05ec"
I20260924 21:12:19.299535 8258 raft_consensus.cc:3037] T e21d0e75f05a4562a92d2615874d8dfb P 205cc24777c74b22b462a666e3216372 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:19.299551 8398 raft_consensus.cc:3037] T e21d0e75f05a4562a92d2615874d8dfb P 82d3aabdbec945738a0b35a8b2db05ec [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:19.300078 8258 raft_consensus.cc:2454] T e21d0e75f05a4562a92d2615874d8dfb P 205cc24777c74b22b462a666e3216372 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e886851288674ad4bf7ba99c9d769e3d in term 1.
I20260924 21:12:19.300189 8398 raft_consensus.cc:2454] T e21d0e75f05a4562a92d2615874d8dfb P 82d3aabdbec945738a0b35a8b2db05ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e886851288674ad4bf7ba99c9d769e3d in term 1.
I20260924 21:12:19.300248 8471 leader_election.cc:302] T e21d0e75f05a4562a92d2615874d8dfb P e886851288674ad4bf7ba99c9d769e3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 205cc24777c74b22b462a666e3216372, e886851288674ad4bf7ba99c9d769e3d; no voters:
I20260924 21:12:19.300344 8613 raft_consensus.cc:2781] T e21d0e75f05a4562a92d2615874d8dfb P e886851288674ad4bf7ba99c9d769e3d [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:19.300423 8613 raft_consensus.cc:686] T e21d0e75f05a4562a92d2615874d8dfb P e886851288674ad4bf7ba99c9d769e3d [term 1 LEADER]: Becoming Leader. State: Replica: e886851288674ad4bf7ba99c9d769e3d, State: Running, Role: LEADER
I20260924 21:12:19.300480 8613 consensus_queue.cc:237] T e21d0e75f05a4562a92d2615874d8dfb P e886851288674ad4bf7ba99c9d769e3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } }
I20260924 21:12:19.300854 8617 tablet_replica.cc:439] T e21d0e75f05a4562a92d2615874d8dfb P e886851288674ad4bf7ba99c9d769e3d: TxnStatusTablet state changed. Reason: New leader e886851288674ad4bf7ba99c9d769e3d. Latest consensus state: current_term: 1 leader_uuid: "e886851288674ad4bf7ba99c9d769e3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } } }
I20260924 21:12:19.300954 8617 tablet_replica.cc:442] T e21d0e75f05a4562a92d2615874d8dfb P e886851288674ad4bf7ba99c9d769e3d: This TxnStatusTablet replica's current role is: LEADER
I20260924 21:12:19.301107 8646 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260924 21:12:19.301224 8646 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260924 21:12:19.301177 8100 catalog_manager.cc:5495] T e21d0e75f05a4562a92d2615874d8dfb P e886851288674ad4bf7ba99c9d769e3d reported cstate change: term changed from 0 to 1, leader changed from <none> to e886851288674ad4bf7ba99c9d769e3d (127.6.210.131). New cstate: current_term: 1 leader_uuid: "e886851288674ad4bf7ba99c9d769e3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } health_report { overall_health: HEALTHY } } }
I20260924 21:12:19.337271 8602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:19.334897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38564 (local address 127.6.210.131:40047)
0924 21:12:19.334952 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:19.334955 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:19.335018 (+ 63us) server_negotiation.cc:407] Connection header received
0924 21:12:19.335043 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:19.335044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:19.335056 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:19.335126 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:12:19.335278 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:19.335745 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:19.336507 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:19.336706 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:19.336783 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:19.336980 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:12:19.337021 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:12:19.337086 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:19.337191 (+ 105us) server_negotiation.cc:299] Negotiation successful
0924 21:12:19.337216 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 21:12:19.337321 8641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:19.334820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:40047 (local address 127.0.0.1:38564)
0924 21:12:19.334897 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:19.334910 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:19.334966 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:19.335134 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:19.335135 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:19.335136 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 21:12:19.335186 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:19.335189 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:19.335829 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:19.335830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:19.336391 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:19.336392 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:19.336782 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:19.336783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:19.336850 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:19.336866 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 21:12:19.337111 (+ 245us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 21:12:19.337122 (+ 11us) client_negotiation.cc:769] Sending connection context
0924 21:12:19.337188 (+ 66us) client_negotiation.cc:240] Negotiation successful
0924 21:12:19.337241 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260924 21:12:19.338442 8398 raft_consensus.cc:1261] T e21d0e75f05a4562a92d2615874d8dfb P 82d3aabdbec945738a0b35a8b2db05ec [term 1 FOLLOWER]: Refusing update from remote peer e886851288674ad4bf7ba99c9d769e3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:19.338438 8258 raft_consensus.cc:1261] T e21d0e75f05a4562a92d2615874d8dfb P 205cc24777c74b22b462a666e3216372 [term 1 FOLLOWER]: Refusing update from remote peer e886851288674ad4bf7ba99c9d769e3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:19.338675 8617 consensus_queue.cc:1035] T e21d0e75f05a4562a92d2615874d8dfb P e886851288674ad4bf7ba99c9d769e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:12:19.338768 8613 consensus_queue.cc:1035] T e21d0e75f05a4562a92d2615874d8dfb P e886851288674ad4bf7ba99c9d769e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:12:19.339251 8614 tablet_replica.cc:439] T e21d0e75f05a4562a92d2615874d8dfb P 82d3aabdbec945738a0b35a8b2db05ec: TxnStatusTablet state changed. Reason: New leader e886851288674ad4bf7ba99c9d769e3d. Latest consensus state: current_term: 1 leader_uuid: "e886851288674ad4bf7ba99c9d769e3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } } }
I20260924 21:12:19.339351 8614 tablet_replica.cc:442] T e21d0e75f05a4562a92d2615874d8dfb P 82d3aabdbec945738a0b35a8b2db05ec: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:12:19.339345 8615 tablet_replica.cc:439] T e21d0e75f05a4562a92d2615874d8dfb P 205cc24777c74b22b462a666e3216372: TxnStatusTablet state changed. Reason: New leader e886851288674ad4bf7ba99c9d769e3d. Latest consensus state: current_term: 1 leader_uuid: "e886851288674ad4bf7ba99c9d769e3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } } }
I20260924 21:12:19.339426 8615 tablet_replica.cc:442] T e21d0e75f05a4562a92d2615874d8dfb P 205cc24777c74b22b462a666e3216372: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:12:19.340061 8617 tablet_replica.cc:439] T e21d0e75f05a4562a92d2615874d8dfb P e886851288674ad4bf7ba99c9d769e3d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e886851288674ad4bf7ba99c9d769e3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } } }
I20260924 21:12:19.340153 8617 tablet_replica.cc:442] T e21d0e75f05a4562a92d2615874d8dfb P e886851288674ad4bf7ba99c9d769e3d: This TxnStatusTablet replica's current role is: LEADER
I20260924 21:12:19.340363 8615 tablet_replica.cc:439] T e21d0e75f05a4562a92d2615874d8dfb P 205cc24777c74b22b462a666e3216372: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e886851288674ad4bf7ba99c9d769e3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } } }
I20260924 21:12:19.340452 8615 tablet_replica.cc:442] T e21d0e75f05a4562a92d2615874d8dfb P 205cc24777c74b22b462a666e3216372: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:12:19.340512 8613 tablet_replica.cc:439] T e21d0e75f05a4562a92d2615874d8dfb P e886851288674ad4bf7ba99c9d769e3d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e886851288674ad4bf7ba99c9d769e3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } } }
I20260924 21:12:19.340585 8613 tablet_replica.cc:442] T e21d0e75f05a4562a92d2615874d8dfb P e886851288674ad4bf7ba99c9d769e3d: This TxnStatusTablet replica's current role is: LEADER
I20260924 21:12:19.340761 8614 tablet_replica.cc:439] T e21d0e75f05a4562a92d2615874d8dfb P 82d3aabdbec945738a0b35a8b2db05ec: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e886851288674ad4bf7ba99c9d769e3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 } } peers { permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 } } peers { permanent_uuid: "e886851288674ad4bf7ba99c9d769e3d" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 40047 } } }
I20260924 21:12:19.340838 8614 tablet_replica.cc:442] T e21d0e75f05a4562a92d2615874d8dfb P 82d3aabdbec945738a0b35a8b2db05ec: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:12:19.346640 8602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:19.344065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:38007 (local address 127.6.210.131:40047)
0924 21:12:19.344120 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:19.344122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:19.344185 (+ 63us) server_negotiation.cc:407] Connection header received
0924 21:12:19.344245 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:19.344246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:19.344252 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:19.344290 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:12:19.344404 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:19.344831 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:19.345886 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:19.346141 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:19.346209 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:19.346407 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:12:19.346432 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:12:19.346472 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:19.346535 (+ 63us) server_negotiation.cc:299] Negotiation successful
0924 21:12:19.346550 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 21:12:19.346705 8590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:19.344003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:40047 (local address 127.6.210.131:38007)
0924 21:12:19.344134 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:19.344146 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:19.344197 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:19.344300 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:19.344301 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:19.344302 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 21:12:19.344344 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:19.344345 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:19.344901 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:19.344902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:19.345798 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:19.345800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:19.346221 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:19.346222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:19.346301 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:19.346317 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 21:12:19.346488 (+ 171us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 21:12:19.346489 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 21:12:19.346535 (+ 46us) client_negotiation.cc:240] Negotiation successful
0924 21:12:19.346577 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91}
I20260924 21:12:19.348631 8398 raft_consensus.cc:1261] T 16c2066b14ec4fbc8d18af768763a9cb P 82d3aabdbec945738a0b35a8b2db05ec [term 1 FOLLOWER]: Refusing update from remote peer e886851288674ad4bf7ba99c9d769e3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:19.348659 8258 raft_consensus.cc:1261] T 16c2066b14ec4fbc8d18af768763a9cb P 205cc24777c74b22b462a666e3216372 [term 1 FOLLOWER]: Refusing update from remote peer e886851288674ad4bf7ba99c9d769e3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:19.348842 8613 consensus_queue.cc:1035] T 16c2066b14ec4fbc8d18af768763a9cb P e886851288674ad4bf7ba99c9d769e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "205cc24777c74b22b462a666e3216372" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:12:19.348919 8617 consensus_queue.cc:1035] T 16c2066b14ec4fbc8d18af768763a9cb P e886851288674ad4bf7ba99c9d769e3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "82d3aabdbec945738a0b35a8b2db05ec" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:12:19.350741 8621 mvcc.cc:205] Tried to move back new op lower bound from 7333004653970833408 to 7333004652902809600. Current Snapshot: MvccSnapshot[applied={T|T < 7333004653970833408}]
I20260924 21:12:19.362413 8099 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:12:19.362514 8099 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:12:19.363106 8099 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 271d3ea1dc374c169abd29787d688cb9: Sending DeleteTablet for 3 replicas of tablet 16c2066b14ec4fbc8d18af768763a9cb
I20260924 21:12:19.363314 8238 tablet_service.cc:1514] Processing DeleteTablet for tablet 16c2066b14ec4fbc8d18af768763a9cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:19 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:45384
I20260924 21:12:19.363376 8378 tablet_service.cc:1514] Processing DeleteTablet for tablet 16c2066b14ec4fbc8d18af768763a9cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:19 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:51146
I20260924 21:12:19.363422 8629 tablet_replica.cc:330] T 16c2066b14ec4fbc8d18af768763a9cb P 205cc24777c74b22b462a666e3216372: stopping tablet replica
I20260924 21:12:19.363466 8629 raft_consensus.cc:2229] T 16c2066b14ec4fbc8d18af768763a9cb P 205cc24777c74b22b462a666e3216372 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:12:19.363469 8631 tablet_replica.cc:330] T 16c2066b14ec4fbc8d18af768763a9cb P 82d3aabdbec945738a0b35a8b2db05ec: stopping tablet replica
I20260924 21:12:19.363498 8629 raft_consensus.cc:2258] T 16c2066b14ec4fbc8d18af768763a9cb P 205cc24777c74b22b462a666e3216372 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:19.363511 8631 raft_consensus.cc:2229] T 16c2066b14ec4fbc8d18af768763a9cb P 82d3aabdbec945738a0b35a8b2db05ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:12:19.363541 8631 raft_consensus.cc:2258] T 16c2066b14ec4fbc8d18af768763a9cb P 82d3aabdbec945738a0b35a8b2db05ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:19.363821 8629 ts_tablet_manager.cc:1905] T 16c2066b14ec4fbc8d18af768763a9cb P 205cc24777c74b22b462a666e3216372: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:19.363934 8631 ts_tablet_manager.cc:1905] T 16c2066b14ec4fbc8d18af768763a9cb P 82d3aabdbec945738a0b35a8b2db05ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:19.363981 8518 tablet_service.cc:1514] Processing DeleteTablet for tablet 16c2066b14ec4fbc8d18af768763a9cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:19 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:38534
I20260924 21:12:19.364063 8630 tablet_replica.cc:330] T 16c2066b14ec4fbc8d18af768763a9cb P e886851288674ad4bf7ba99c9d769e3d: stopping tablet replica
I20260924 21:12:19.364106 8630 raft_consensus.cc:2229] T 16c2066b14ec4fbc8d18af768763a9cb P e886851288674ad4bf7ba99c9d769e3d [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:12:19.364151 8630 raft_consensus.cc:2258] T 16c2066b14ec4fbc8d18af768763a9cb P e886851288674ad4bf7ba99c9d769e3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:19.365437 8630 ts_tablet_manager.cc:1905] T 16c2066b14ec4fbc8d18af768763a9cb P e886851288674ad4bf7ba99c9d769e3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:19.365583 8631 ts_tablet_manager.cc:1918] T 16c2066b14ec4fbc8d18af768763a9cb P 82d3aabdbec945738a0b35a8b2db05ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 21:12:19.365640 8631 log.cc:1196] T 16c2066b14ec4fbc8d18af768763a9cb P 82d3aabdbec945738a0b35a8b2db05ec: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-1/wal/wals/16c2066b14ec4fbc8d18af768763a9cb
I20260924 21:12:19.365631 8629 ts_tablet_manager.cc:1918] T 16c2066b14ec4fbc8d18af768763a9cb P 205cc24777c74b22b462a666e3216372: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 21:12:19.365690 8629 log.cc:1196] T 16c2066b14ec4fbc8d18af768763a9cb P 205cc24777c74b22b462a666e3216372: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-0/wal/wals/16c2066b14ec4fbc8d18af768763a9cb
I20260924 21:12:19.365860 8631 ts_tablet_manager.cc:1939] T 16c2066b14ec4fbc8d18af768763a9cb P 82d3aabdbec945738a0b35a8b2db05ec: Deleting consensus metadata
I20260924 21:12:19.365907 8629 ts_tablet_manager.cc:1939] T 16c2066b14ec4fbc8d18af768763a9cb P 205cc24777c74b22b462a666e3216372: Deleting consensus metadata
I20260924 21:12:19.366336 8087 catalog_manager.cc:4841] TS 205cc24777c74b22b462a666e3216372 (127.6.210.129:42409): tablet 16c2066b14ec4fbc8d18af768763a9cb (table test-table [id=1f89b4ffc1b047819dce8941d977e6a1]) successfully deleted
I20260924 21:12:19.366438 8087 catalog_manager.cc:4841] TS 82d3aabdbec945738a0b35a8b2db05ec (127.6.210.130:34611): tablet 16c2066b14ec4fbc8d18af768763a9cb (table test-table [id=1f89b4ffc1b047819dce8941d977e6a1]) successfully deleted
I20260924 21:12:19.367321 8630 ts_tablet_manager.cc:1918] T 16c2066b14ec4fbc8d18af768763a9cb P e886851288674ad4bf7ba99c9d769e3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 21:12:19.367380 8630 log.cc:1196] T 16c2066b14ec4fbc8d18af768763a9cb P e886851288674ad4bf7ba99c9d769e3d: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284333163253-6986-0/minicluster-data/ts-2/wal/wals/16c2066b14ec4fbc8d18af768763a9cb
I20260924 21:12:19.367612 8630 ts_tablet_manager.cc:1939] T 16c2066b14ec4fbc8d18af768763a9cb P e886851288674ad4bf7ba99c9d769e3d: Deleting consensus metadata
I20260924 21:12:19.367901 8087 catalog_manager.cc:4841] TS e886851288674ad4bf7ba99c9d769e3d (127.6.210.131:40047): tablet 16c2066b14ec4fbc8d18af768763a9cb (table test-table [id=1f89b4ffc1b047819dce8941d977e6a1]) successfully deleted
Sep 24 21:12:19 dist-test-slave-q276 krb5kdc[8056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284338, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.210.129@KRBTEST.COM
I20260924 21:12:19.375378 8606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:19.365449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45394 (local address 127.6.210.129:42409)
0924 21:12:19.365490 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:19.365492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:19.366317 (+ 825us) server_negotiation.cc:407] Connection header received
0924 21:12:19.366447 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:19.366448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:19.366458 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:19.366489 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:19.367269 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:19.368023 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:19.369043 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:19.369219 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:19.369265 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:19.372414 (+ 3149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:19.372423 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:19.372423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:19.372430 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:19.374148 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:19.374534 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:19.374538 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:19.374539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:19.374620 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:19.374827 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:19.374828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:19.374828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:19.374975 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:19.375132 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:19.375192 (+ 60us) server_negotiation.cc:299] Negotiation successful
0924 21:12:19.375276 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260924 21:12:19.375793 8298 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45394
I20260924 21:12:19.382746 8154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:19.376864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34460 (local address 127.6.210.190:39053)
0924 21:12:19.376913 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:19.376915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:19.376924 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:12:19.376988 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:19.376989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:19.376999 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:19.377042 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:19.377506 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:19.377992 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:19.378740 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:19.378943 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:19.379021 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:19.380144 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:19.380145 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:19.380145 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:19.380153 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:19.381542 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:19.381866 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:19.381866 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:19.381867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:19.381923 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:19.382179 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:19.382180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:19.382180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:19.382342 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:19.382451 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:19.382573 (+ 122us) server_negotiation.cc:299] Negotiation successful
0924 21:12:19.382644 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260924 21:12:19.382992 8099 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34460
I20260924 21:12:19.383652 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 8173
I20260924 21:12:19.389546 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 8313
I20260924 21:12:19.396330 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 8453
I20260924 21:12:19.402072 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 8069
2026-09-24T21:12:19Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2362 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:12:19 dist-test-slave-q276 krb5kdc[8663](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:12:19 dist-test-slave-q276 krb5kdc[8663](info): set up 2 sockets
Sep 24 21:12:19 dist-test-slave-q276 krb5kdc[8663](info): commencing operation
krb5kdc: starting...
W20260924 21:12:20.454977 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.003s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:12:20 dist-test-slave-q276 krb5kdc[8663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:12:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:12:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.210.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
I20260924 21:12:20.576633 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:36861
--webserver_interface=127.6.210.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:41679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:36861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:20.655081 8676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:20.655319 8676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:20.655357 8676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:20.656522 8676 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:12:20.656594 8676 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260924 21:12:20.656617 8676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:20.656639 8676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:20.656668 8676 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:12:20.656692 8676 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:12:20.658245 8676 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:41679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:36861
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:36861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:20.658524 8676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:20.658742 8676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:20.663528 8681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:20.663522 8684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:20.663667 8676 server_base.cc:1034] running on GCE node
W20260924 21:12:20.663522 8682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:20.663990 8676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:20.664314 8676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:20.665493 8676 hybrid_clock.cc:648] HybridClock initialized: now 1790284340665474 us; error 40 us; skew 500 ppm
Sep 24 21:12:20 dist-test-slave-q276 krb5kdc[8663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284340, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:20.668092 8676 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:12:20.668538 8676 webserver.cc:460] Webserver started at http://127.6.210.190:38961/ using document root <none> and password file <none>
I20260924 21:12:20.668737 8676 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:20.668778 8676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:20.668866 8676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:20.669862 8676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "14bb132e638b4ce3914b44d1e001055a"
format_stamp: "Formatted at 2026-09-24 21:12:20 on dist-test-slave-q276"
server_key: "ab132cc99554b743d21e26d1ac34ad3a"
server_key_iv: "649ae22212de0ca29dea87b9b94e166b"
server_key_version: "encryptionkey@0"
I20260924 21:12:20.670152 8676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "14bb132e638b4ce3914b44d1e001055a"
format_stamp: "Formatted at 2026-09-24 21:12:20 on dist-test-slave-q276"
server_key: "ab132cc99554b743d21e26d1ac34ad3a"
server_key_iv: "649ae22212de0ca29dea87b9b94e166b"
server_key_version: "encryptionkey@0"
I20260924 21:12:20.671432 8676 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:12:20.672050 8691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:20.672220 8676 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:12:20.672300 8676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "14bb132e638b4ce3914b44d1e001055a"
format_stamp: "Formatted at 2026-09-24 21:12:20 on dist-test-slave-q276"
server_key: "ab132cc99554b743d21e26d1ac34ad3a"
server_key_iv: "649ae22212de0ca29dea87b9b94e166b"
server_key_version: "encryptionkey@0"
I20260924 21:12:20.672353 8676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:20.694777 8676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:20.697099 8676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:20.697211 8676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:20.701577 8676 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:36861
I20260924 21:12:20.701639 8753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:36861 every 8 connection(s)
I20260924 21:12:20.701977 8676 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:12:20.702729 8754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:20.704910 8754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14bb132e638b4ce3914b44d1e001055a: Bootstrap starting.
I20260924 21:12:20.705485 8754 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14bb132e638b4ce3914b44d1e001055a: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:20.705794 8754 log.cc:824] T 00000000000000000000000000000000 P 14bb132e638b4ce3914b44d1e001055a: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:20.706501 8754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14bb132e638b4ce3914b44d1e001055a: No bootstrap required, opened a new log
I20260924 21:12:20.708552 8754 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14bb132e638b4ce3914b44d1e001055a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14bb132e638b4ce3914b44d1e001055a" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36861 } }
I20260924 21:12:20.708709 8754 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14bb132e638b4ce3914b44d1e001055a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:20.708756 8754 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14bb132e638b4ce3914b44d1e001055a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14bb132e638b4ce3914b44d1e001055a, State: Initialized, Role: FOLLOWER
I20260924 21:12:20.708943 8754 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14bb132e638b4ce3914b44d1e001055a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14bb132e638b4ce3914b44d1e001055a" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36861 } }
I20260924 21:12:20.709050 8754 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14bb132e638b4ce3914b44d1e001055a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:12:20.709097 8754 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14bb132e638b4ce3914b44d1e001055a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:12:20.709148 8754 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14bb132e638b4ce3914b44d1e001055a [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:20.709662 8754 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14bb132e638b4ce3914b44d1e001055a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14bb132e638b4ce3914b44d1e001055a" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36861 } }
I20260924 21:12:20.709779 8754 leader_election.cc:302] T 00000000000000000000000000000000 P 14bb132e638b4ce3914b44d1e001055a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14bb132e638b4ce3914b44d1e001055a; no voters:
I20260924 21:12:20.709951 8754 leader_election.cc:288] T 00000000000000000000000000000000 P 14bb132e638b4ce3914b44d1e001055a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:12:20.710011 8757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14bb132e638b4ce3914b44d1e001055a [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:20.710207 8754 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14bb132e638b4ce3914b44d1e001055a [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:12:20.710239 8757 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14bb132e638b4ce3914b44d1e001055a [term 1 LEADER]: Becoming Leader. State: Replica: 14bb132e638b4ce3914b44d1e001055a, State: Running, Role: LEADER
I20260924 21:12:20.710387 8757 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14bb132e638b4ce3914b44d1e001055a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14bb132e638b4ce3914b44d1e001055a" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36861 } }
I20260924 21:12:20.710800 8758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14bb132e638b4ce3914b44d1e001055a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14bb132e638b4ce3914b44d1e001055a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14bb132e638b4ce3914b44d1e001055a" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36861 } } }
I20260924 21:12:20.710813 8759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14bb132e638b4ce3914b44d1e001055a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14bb132e638b4ce3914b44d1e001055a. Latest consensus state: current_term: 1 leader_uuid: "14bb132e638b4ce3914b44d1e001055a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14bb132e638b4ce3914b44d1e001055a" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36861 } } }
I20260924 21:12:20.711017 8758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14bb132e638b4ce3914b44d1e001055a [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:20.711023 8759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14bb132e638b4ce3914b44d1e001055a [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:20.711272 8766 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:12:20.711290 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 8676
I20260924 21:12:20.711400 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:12:20.711620 6986 external_mini_cluster.cc:1428] Setting key 813906e3bf7e9d69f8340cfb861e8710
I20260924 21:12:20.712055 8766 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:12:20.713626 8766 catalog_manager.cc:1347] Generated new cluster ID: edcb11d7f3f24e4ebf6330c42fd8f76c
I20260924 21:12:20.713718 8766 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 21:12:20 dist-test-slave-q276 krb5kdc[8663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:20.724259 8776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:20.712290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47762 (local address 127.6.210.190:36861)
0924 21:12:20.712661 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:20.712671 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:20.712703 (+ 32us) server_negotiation.cc:407] Connection header received
0924 21:12:20.712818 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:20.712826 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:20.712879 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:20.713031 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:20.713892 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:20.715264 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:20.716408 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:20.717002 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:20.717094 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:20.720131 (+ 3037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:20.720137 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:20.720137 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:20.720150 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:20.722450 (+ 2300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:20.722849 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:20.722853 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:20.722853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:20.722991 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:20.723213 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:20.723214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:20.723215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:20.723575 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:20.723774 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:20.723866 (+ 92us) server_negotiation.cc:299] Negotiation successful
0924 21:12:20.724025 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":276,"threads_started":1}
I20260924 21:12:20.734702 8766 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:12:20.735499 8766 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:12:20.747887 8766 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14bb132e638b4ce3914b44d1e001055a: Generated new TSK 0
I20260924 21:12:20.748116 8766 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.210.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
I20260924 21:12:20.777417 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.129:0
--local_ip_for_outbound_sockets=127.6.210.129
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:36861
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:41679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:20.861758 8780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:20.862031 8780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:20.862059 8780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:20.863291 8780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:20.863332 8780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:20.863353 8780 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260924 21:12:20.863364 8780 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 21:12:20.863384 8780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.129
I20260924 21:12:20.864817 8780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:41679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.210.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.210.190:36861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.129
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:20.865092 8780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:20.865321 8780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:20.870414 8786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:20.870471 8785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:20.870637 8780 server_base.cc:1034] running on GCE node
W20260924 21:12:20.870414 8788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:20.871078 8780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:20.871351 8780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:20.872576 8780 hybrid_clock.cc:648] HybridClock initialized: now 1790284340872551 us; error 40 us; skew 500 ppm
Sep 24 21:12:20 dist-test-slave-q276 krb5kdc[8663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284340, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:20.875383 8780 init.cc:377] Logged in from keytab as kudu/127.6.210.129@KRBTEST.COM (short username kudu)
I20260924 21:12:20.875864 8780 webserver.cc:460] Webserver started at http://127.6.210.129:42091/ using document root <none> and password file <none>
I20260924 21:12:20.876132 8780 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:20.876207 8780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:20.876396 8780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:20.877599 8780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-0/data/instance:
uuid: "30eebf7f3eaf46398365efa962c43c36"
format_stamp: "Formatted at 2026-09-24 21:12:20 on dist-test-slave-q276"
server_key: "e0a8765d8c92a9448e0f1f18da5948a6"
server_key_iv: "4db6db83e12e1e312f48bf7b533ed4a5"
server_key_version: "encryptionkey@0"
I20260924 21:12:20.878021 8780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance:
uuid: "30eebf7f3eaf46398365efa962c43c36"
format_stamp: "Formatted at 2026-09-24 21:12:20 on dist-test-slave-q276"
server_key: "e0a8765d8c92a9448e0f1f18da5948a6"
server_key_iv: "4db6db83e12e1e312f48bf7b533ed4a5"
server_key_version: "encryptionkey@0"
I20260924 21:12:20.879595 8780 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:12:20.880437 8795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:20.880692 8780 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:20.880818 8780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-0/wal
uuid: "30eebf7f3eaf46398365efa962c43c36"
format_stamp: "Formatted at 2026-09-24 21:12:20 on dist-test-slave-q276"
server_key: "e0a8765d8c92a9448e0f1f18da5948a6"
server_key_iv: "4db6db83e12e1e312f48bf7b533ed4a5"
server_key_version: "encryptionkey@0"
I20260924 21:12:20.880942 8780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:20.887814 8780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:20.889591 8780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:20.889756 8780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:20.890234 8780 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:20.890348 8780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:20.890396 8780 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:20.890424 8780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:20.897153 8780 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.129:39809
Sep 24 21:12:20 dist-test-slave-q276 krb5kdc[8663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284340, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:20.897236 8910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.129:39809 every 8 connection(s)
I20260924 21:12:20.897549 8780 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
I20260924 21:12:20.900750 8776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:20.890771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:33065 (local address 127.6.210.190:36861)
0924 21:12:20.890828 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:20.890830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:20.891061 (+ 231us) server_negotiation.cc:407] Connection header received
0924 21:12:20.891573 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:20.891574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:20.891587 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:20.891646 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:20.892426 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:20.892969 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:20.894556 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:20.894938 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:20.895028 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:20.897712 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:20.897714 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:20.897714 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:20.897727 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:20.899349 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:20.899778 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:20.899779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:20.899779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:20.899861 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:20.900180 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:20.900181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:20.900181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:20.900292 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:20.900349 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:20.900612 (+ 263us) server_negotiation.cc:299] Negotiation successful
0924 21:12:20.900686 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 21:12:20.900915 8804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:20.890759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36861 (local address 127.6.210.129:33065)
0924 21:12:20.890917 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:20.890950 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:20.891465 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:20.891710 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:20.891714 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:20.892066 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:20.892339 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:20.892344 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:20.893073 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:20.893074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:20.894452 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:20.894455 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:20.895048 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:20.895049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:20.895191 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:20.895731 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:20.895741 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:20.897608 (+ 1867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:20.899490 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:20.899495 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:20.899497 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:20.899704 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:20.899937 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:20.899938 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:20.899938 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:20.900102 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:20.900350 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:20.900354 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:20.900463 (+ 109us) client_negotiation.cc:769] Sending connection context
0924 21:12:20.900588 (+ 125us) client_negotiation.cc:240] Negotiation successful
0924 21:12:20.900724 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":91,"threads_started":1}
I20260924 21:12:20.901449 8911 heartbeater.cc:347] Connected to a master server at 127.6.210.190:36861
I20260924 21:12:20.901538 8911 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:20.901721 8911 heartbeater.cc:511] Master 127.6.210.190:36861 requested a full tablet report, sending...
I20260924 21:12:20.902251 8707 ts_manager.cc:194] Registered new tserver with Master: 30eebf7f3eaf46398365efa962c43c36 (127.6.210.129:39809)
I20260924 21:12:20.903258 8707 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.129@KRBTEST.COM'} at 127.6.210.129:33065
I20260924 21:12:20.903589 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 8780
I20260924 21:12:20.903700 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance
I20260924 21:12:20.903882 6986 external_mini_cluster.cc:1428] Setting key ca825c77a6b8836ea4253532f073628c
I20260924 21:12:20.908324 8776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:20.902545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:57785 (local address 127.6.210.190:36861)
0924 21:12:20.902599 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:20.902601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:20.902633 (+ 32us) server_negotiation.cc:407] Connection header received
0924 21:12:20.902711 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:20.902712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:20.902719 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:20.902762 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:20.903091 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:20.903673 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:20.904600 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:20.904878 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:20.904948 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:20.905882 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:20.905884 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:20.905884 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:20.905891 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:20.907206 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:20.907562 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:20.907563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:20.907564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:20.907616 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:20.907811 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:20.907812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:20.907813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:20.907922 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:20.908009 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:20.908162 (+ 153us) server_negotiation.cc:299] Negotiation successful
0924 21:12:20.908237 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260924 21:12:20.908341 8917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:20.902428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36861 (local address 127.6.210.129:57785)
0924 21:12:20.902572 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:20.902584 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:20.902647 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:20.902767 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:20.902768 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:20.902960 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:20.903024 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:20.903025 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:20.903744 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:20.903745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:20.904507 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:20.904509 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:20.904948 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:20.904949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:20.905030 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:20.905509 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:20.905513 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:20.905797 (+ 284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:20.907304 (+ 1507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:20.907305 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:20.907306 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:20.907478 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:20.907688 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:20.907689 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:20.907690 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:20.907752 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:20.908003 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:20.908004 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:20.908057 (+ 53us) client_negotiation.cc:769] Sending connection context
0924 21:12:20.908138 (+ 81us) client_negotiation.cc:240] Negotiation successful
0924 21:12:20.908237 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":113,"thread_start_us":70,"threads_started":1}
WARNING: no policy specified for kudu/127.6.210.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
I20260924 21:12:20.929566 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.130:0
--local_ip_for_outbound_sockets=127.6.210.130
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:36861
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:41679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:21.013760 8920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:21.014029 8920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:21.014057 8920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:21.015233 8920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:21.015275 8920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:21.015295 8920 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260924 21:12:21.015307 8920 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 21:12:21.015324 8920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.130
I20260924 21:12:21.016709 8920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:41679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.210.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.210.190:36861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.130
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:21.016893 8920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:21.017069 8920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:21.022089 8926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:21.022107 8928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:21.022097 8925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:21.022380 8920 server_base.cc:1034] running on GCE node
I20260924 21:12:21.022596 8920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:21.022856 8920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:21.024060 8920 hybrid_clock.cc:648] HybridClock initialized: now 1790284341024003 us; error 56 us; skew 500 ppm
Sep 24 21:12:21 dist-test-slave-q276 krb5kdc[8663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284341, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:21.027036 8920 init.cc:377] Logged in from keytab as kudu/127.6.210.130@KRBTEST.COM (short username kudu)
I20260924 21:12:21.027557 8920 webserver.cc:460] Webserver started at http://127.6.210.130:41523/ using document root <none> and password file <none>
I20260924 21:12:21.027773 8920 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:21.027827 8920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:21.027928 8920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:21.029063 8920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-1/data/instance:
uuid: "3ae7d6322d6d4d3f83a208fa4d76c379"
format_stamp: "Formatted at 2026-09-24 21:12:21 on dist-test-slave-q276"
server_key: "84c820c8b14e47c7402ed8229c9a9320"
server_key_iv: "9435d70443381df835c49aaa7065b403"
server_key_version: "encryptionkey@0"
I20260924 21:12:21.029444 8920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance:
uuid: "3ae7d6322d6d4d3f83a208fa4d76c379"
format_stamp: "Formatted at 2026-09-24 21:12:21 on dist-test-slave-q276"
server_key: "84c820c8b14e47c7402ed8229c9a9320"
server_key_iv: "9435d70443381df835c49aaa7065b403"
server_key_version: "encryptionkey@0"
I20260924 21:12:21.030994 8920 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:12:21.031889 8935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:21.032096 8920 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:21.032217 8920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-1/wal
uuid: "3ae7d6322d6d4d3f83a208fa4d76c379"
format_stamp: "Formatted at 2026-09-24 21:12:21 on dist-test-slave-q276"
server_key: "84c820c8b14e47c7402ed8229c9a9320"
server_key_iv: "9435d70443381df835c49aaa7065b403"
server_key_version: "encryptionkey@0"
I20260924 21:12:21.032301 8920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:21.063230 8920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:21.065886 8920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:21.066071 8920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:21.066722 8920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:21.066788 8920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:21.066826 8920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:21.066859 8920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 21:12:21 dist-test-slave-q276 krb5kdc[8663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284341, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:21.074685 8920 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.130:41407
I20260924 21:12:21.075100 9050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.130:41407 every 8 connection(s)
I20260924 21:12:21.075151 8920 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
I20260924 21:12:21.078068 8776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:21.067222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:49467 (local address 127.6.210.190:36861)
0924 21:12:21.067267 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:21.067269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:21.067636 (+ 367us) server_negotiation.cc:407] Connection header received
0924 21:12:21.068192 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:21.068194 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:21.068208 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:21.068259 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:21.068954 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.069776 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.071681 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.071896 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.071998 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:21.074685 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:21.074687 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:21.074687 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:21.074695 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:21.076643 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:21.077108 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:21.077109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:21.077109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:21.077185 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:21.077449 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:21.077450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:21.077450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:21.077573 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:21.077640 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:21.077878 (+ 238us) server_negotiation.cc:299] Negotiation successful
0924 21:12:21.078004 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 21:12:21.078187 8944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:21.067220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36861 (local address 127.6.210.130:49467)
0924 21:12:21.067396 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:21.067450 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:21.068071 (+ 621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:21.068331 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:21.068334 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:21.068669 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:21.068856 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:21.068861 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.069895 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.069896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:21.071561 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:21.071565 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.072023 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.072024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:21.072157 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:21.072660 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:21.072669 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:21.074582 (+ 1913us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:21.076738 (+ 2156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:21.076744 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:21.076746 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:21.076968 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:21.077257 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:21.077258 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:21.077258 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:21.077358 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:21.077642 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:21.077646 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:21.077762 (+ 116us) client_negotiation.cc:769] Sending connection context
0924 21:12:21.077847 (+ 85us) client_negotiation.cc:240] Negotiation successful
0924 21:12:21.077967 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":91,"threads_started":1}
I20260924 21:12:21.078720 9051 heartbeater.cc:347] Connected to a master server at 127.6.210.190:36861
I20260924 21:12:21.078822 9051 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:21.079008 9051 heartbeater.cc:511] Master 127.6.210.190:36861 requested a full tablet report, sending...
I20260924 21:12:21.079458 8707 ts_manager.cc:194] Registered new tserver with Master: 3ae7d6322d6d4d3f83a208fa4d76c379 (127.6.210.130:41407)
I20260924 21:12:21.079895 8707 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.130@KRBTEST.COM'} at 127.6.210.130:49467
I20260924 21:12:21.085057 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 8920
I20260924 21:12:21.085139 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance
I20260924 21:12:21.085317 6986 external_mini_cluster.cc:1428] Setting key aee20ae29b646ded6a04f208b6b0b90a
I20260924 21:12:21.085549 8776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:21.079907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:51333 (local address 127.6.210.190:36861)
0924 21:12:21.079951 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:21.079953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:21.080024 (+ 71us) server_negotiation.cc:407] Connection header received
0924 21:12:21.080085 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:21.080086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:21.080096 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:21.080156 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:21.080440 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.080901 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.081626 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.081825 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.081909 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:21.082882 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:21.082884 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:21.082884 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:21.082891 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:21.084321 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:21.084707 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:21.084709 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:21.084709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:21.084762 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:21.084974 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:21.084975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:21.084975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:21.085140 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:21.085217 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:21.085382 (+ 165us) server_negotiation.cc:299] Negotiation successful
0924 21:12:21.085460 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 21:12:21.085577 9057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:21.079819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36861 (local address 127.6.210.130:51333)
0924 21:12:21.079961 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:21.079972 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:21.080036 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:21.080160 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:21.080160 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:21.080302 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:21.080364 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:21.080365 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.080998 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.080999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:21.081529 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:21.081531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.081910 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.081911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:21.081992 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:21.082465 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:21.082469 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:21.082776 (+ 307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:21.084427 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:21.084428 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:21.084428 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:21.084603 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:21.084845 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:21.084846 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:21.084846 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:21.084906 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:21.085243 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:21.085244 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:21.085281 (+ 37us) client_negotiation.cc:769] Sending connection context
0924 21:12:21.085357 (+ 76us) client_negotiation.cc:240] Negotiation successful
0924 21:12:21.085476 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
WARNING: no policy specified for kudu/127.6.210.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
I20260924 21:12:21.108963 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.131:0
--local_ip_for_outbound_sockets=127.6.210.131
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:36861
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:41679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:21.189337 9060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:21.189551 9060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:21.189575 9060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:21.190750 9060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:21.190791 9060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:21.190812 9060 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260924 21:12:21.190863 9060 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 21:12:21.190888 9060 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.131
I20260924 21:12:21.192325 9060 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:41679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.210.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.210.190:36861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.131
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:21.192569 9060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:21.192801 9060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:21.197680 9068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:21.197737 9060 server_base.cc:1034] running on GCE node
W20260924 21:12:21.197710 9066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:21.197710 9065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:21.198012 9060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:21.198298 9060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:21.199447 9060 hybrid_clock.cc:648] HybridClock initialized: now 1790284341199435 us; error 29 us; skew 500 ppm
Sep 24 21:12:21 dist-test-slave-q276 krb5kdc[8663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284341, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:21.202028 9060 init.cc:377] Logged in from keytab as kudu/127.6.210.131@KRBTEST.COM (short username kudu)
I20260924 21:12:21.202500 9060 webserver.cc:460] Webserver started at http://127.6.210.131:42587/ using document root <none> and password file <none>
I20260924 21:12:21.202716 9060 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:21.202787 9060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:21.202939 9060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:21.204027 9060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-2/data/instance:
uuid: "2fdf8641dc49447db2d5e11d735c0178"
format_stamp: "Formatted at 2026-09-24 21:12:21 on dist-test-slave-q276"
server_key: "c41e756e2c2bc06bbc90226f00388b9e"
server_key_iv: "f2da5628c5a047ca937a71a520ab537e"
server_key_version: "encryptionkey@0"
I20260924 21:12:21.204384 9060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance:
uuid: "2fdf8641dc49447db2d5e11d735c0178"
format_stamp: "Formatted at 2026-09-24 21:12:21 on dist-test-slave-q276"
server_key: "c41e756e2c2bc06bbc90226f00388b9e"
server_key_iv: "f2da5628c5a047ca937a71a520ab537e"
server_key_version: "encryptionkey@0"
I20260924 21:12:21.205713 9060 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:12:21.206445 9075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:21.206607 9060 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:12:21.206689 9060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-2/wal
uuid: "2fdf8641dc49447db2d5e11d735c0178"
format_stamp: "Formatted at 2026-09-24 21:12:21 on dist-test-slave-q276"
server_key: "c41e756e2c2bc06bbc90226f00388b9e"
server_key_iv: "f2da5628c5a047ca937a71a520ab537e"
server_key_version: "encryptionkey@0"
I20260924 21:12:21.206740 9060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:21.220018 9060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:21.222352 9060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:21.222465 9060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:21.222919 9060 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:21.222958 9060 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:21.222985 9060 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:21.222999 9060 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:21.229686 9060 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.131:37881
I20260924 21:12:21.229735 9190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.131:37881 every 8 connection(s)
I20260924 21:12:21.230070 9060 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
Sep 24 21:12:21 dist-test-slave-q276 krb5kdc[8663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284341, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:21.232995 8776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:21.223460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:57853 (local address 127.6.210.190:36861)
0924 21:12:21.223534 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:21.223536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:21.223754 (+ 218us) server_negotiation.cc:407] Connection header received
0924 21:12:21.224420 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:21.224421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:21.224436 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:21.224514 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:21.225368 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.226218 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.227815 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.228169 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.228256 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:21.230723 (+ 2467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:21.230725 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:21.230725 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:21.230733 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:21.231876 (+ 1143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:21.232252 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:21.232252 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:21.232253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:21.232301 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:21.232513 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:21.232514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:21.232514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:21.232613 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:21.232662 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:21.232867 (+ 205us) server_negotiation.cc:299] Negotiation successful
0924 21:12:21.232927 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260924 21:12:21.233273 9084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:21.223481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36861 (local address 127.6.210.131:57853)
0924 21:12:21.223633 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:21.223667 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:21.224292 (+ 625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:21.224603 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:21.224607 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:21.224963 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:21.225254 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:21.225260 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.226350 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.226351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:21.227710 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:21.227713 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.228261 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.228262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:21.228401 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:21.228989 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:21.229003 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:21.230635 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:21.231945 (+ 1310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:21.231950 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:21.231951 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:21.232180 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:21.232357 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:21.232358 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:21.232358 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:21.232452 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:21.232666 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:21.232670 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:21.232776 (+ 106us) client_negotiation.cc:769] Sending connection context
0924 21:12:21.232855 (+ 79us) client_negotiation.cc:240] Negotiation successful
0924 21:12:21.232977 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":92,"threads_started":1}
I20260924 21:12:21.233723 9191 heartbeater.cc:347] Connected to a master server at 127.6.210.190:36861
I20260924 21:12:21.233808 9191 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:21.234030 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 9060
I20260924 21:12:21.234061 9191 heartbeater.cc:511] Master 127.6.210.190:36861 requested a full tablet report, sending...
I20260924 21:12:21.234122 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance
I20260924 21:12:21.234337 6986 external_mini_cluster.cc:1428] Setting key ee345f440601ea4196ba08452a12a1b4
I20260924 21:12:21.234439 8707 ts_manager.cc:194] Registered new tserver with Master: 2fdf8641dc49447db2d5e11d735c0178 (127.6.210.131:37881)
I20260924 21:12:21.234854 8707 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.131@KRBTEST.COM'} at 127.6.210.131:57853
I20260924 21:12:21.235054 6986 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:12:21.241185 9197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:21.235133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36861 (local address 127.6.210.131:55899)
0924 21:12:21.235260 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:21.235270 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:21.235321 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:21.235427 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:21.235427 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:21.235654 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:21.235735 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:21.235737 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.236425 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.236426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:21.237103 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:21.237105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.237478 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.237486 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:21.237562 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:21.237993 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:21.237998 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:21.238355 (+ 357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:21.240071 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:21.240072 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:21.240073 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:21.240307 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:21.240518 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:21.240519 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:21.240519 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:21.240588 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:21.240809 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:21.240810 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:21.240869 (+ 59us) client_negotiation.cc:769] Sending connection context
0924 21:12:21.240942 (+ 73us) client_negotiation.cc:240] Negotiation successful
0924 21:12:21.241058 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":63,"threads_started":1}
I20260924 21:12:21.241197 8776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:21.235281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:55899 (local address 127.6.210.190:36861)
0924 21:12:21.235317 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:21.235318 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:21.235326 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:12:21.235368 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:21.235368 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:21.235377 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:21.235425 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:21.235803 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.236337 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.237196 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.237393 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.237477 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:21.238453 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:21.238454 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:21.238454 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:21.238461 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:21.239988 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:21.240388 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:21.240388 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:21.240389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:21.240436 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:21.240648 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:21.240649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:21.240649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:21.240749 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:21.240798 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:21.240995 (+ 197us) server_negotiation.cc:299] Negotiation successful
0924 21:12:21.241078 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Sep 24 21:12:21 dist-test-slave-q276 krb5kdc[8663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 21:12:21 dist-test-slave-q276 krb5kdc[8663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:21.257995 8776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:21.251279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47766 (local address 127.6.210.190:36861)
0924 21:12:21.251338 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:21.251341 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:21.251360 (+ 19us) server_negotiation.cc:407] Connection header received
0924 21:12:21.251390 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:21.251391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:21.251398 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:21.251459 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:21.252059 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.252541 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.253273 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.253462 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.253538 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:21.255716 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:21.255718 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:21.255718 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:21.255725 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:21.256886 (+ 1161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:21.257257 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:21.257258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:21.257259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:21.257313 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:21.257510 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:21.257511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:21.257512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:21.257615 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:21.257721 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:21.257817 (+ 96us) server_negotiation.cc:299] Negotiation successful
0924 21:12:21.257894 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 21:12:21.259686 8707 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47766:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:12:21.261338 8707 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:12:21.268049 9208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:21.263606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:37881 (local address 127.0.0.1:52186)
0924 21:12:21.263708 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:21.263750 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:21.263808 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:21.264306 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:21.264309 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:21.264311 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:21.264375 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:21.264377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.265679 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.265680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:21.266737 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:21.266739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.267682 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.267683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:21.267808 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:21.267812 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:12:21.267869 (+ 57us) client_negotiation.cc:240] Negotiation successful
0924 21:12:21.267946 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":46,"thread_start_us":60,"threads_started":1}
I20260924 21:12:21.268056 9209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:21.263612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:41407 (local address 127.0.0.1:43062)
0924 21:12:21.263725 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:21.263752 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:21.263810 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:21.264529 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:21.264530 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:21.264532 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:21.264573 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:21.264574 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.265419 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.265420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:21.266714 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:21.266717 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.267684 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.267684 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:21.267782 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:21.267795 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 21:12:21.267870 (+ 75us) client_negotiation.cc:240] Negotiation successful
0924 21:12:21.267950 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":32,"thread_start_us":75,"threads_started":1}
I20260924 21:12:21.268069 9212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:21.263979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52186 (local address 127.6.210.131:37881)
0924 21:12:21.264128 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:21.264133 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:21.264144 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:12:21.264184 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:21.264186 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:21.264204 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:21.264289 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:21.264463 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.265581 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.266815 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.267571 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.267710 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:21.267777 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:21.267894 (+ 117us) server_negotiation.cc:299] Negotiation successful
0924 21:12:21.267937 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
I20260924 21:12:21.268524 9211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:21.263802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43062 (local address 127.6.210.130:41407)
0924 21:12:21.264180 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:21.264184 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:21.264192 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:12:21.264391 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:21.264394 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:21.264413 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:21.264639 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:21.264684 (+ 45us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.265333 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.266816 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.267585 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.268075 (+ 490us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:21.268344 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:21.268366 (+ 22us) server_negotiation.cc:299] Negotiation successful
0924 21:12:21.268400 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":82,"threads_started":1}
I20260924 21:12:21.268818 9210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:21.263806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:39809 (local address 127.0.0.1:46878)
0924 21:12:21.264002 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:21.264011 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:21.264057 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:21.264389 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:21.264390 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:21.264392 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:21.264442 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:21.264444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.266825 (+ 2381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.266825 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:21.267568 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:21.267569 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.268617 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.268618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:21.268674 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:21.268678 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:12:21.268723 (+ 45us) client_negotiation.cc:240] Negotiation successful
0924 21:12:21.268759 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":70,"threads_started":1}
I20260924 21:12:21.269068 9213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:21.264005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46878 (local address 127.6.210.129:39809)
0924 21:12:21.264177 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:21.264182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:21.264189 (+ 7us) server_negotiation.cc:407] Connection header received
0924 21:12:21.264222 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:21.264223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:21.264235 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:21.264325 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:21.264520 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.265587 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.267671 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.268463 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.268528 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:21.268583 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:21.268919 (+ 336us) server_negotiation.cc:299] Negotiation successful
0924 21:12:21.268961 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":70,"threads_started":1}
I20260924 21:12:21.268981 9125 tablet_service.cc:1467] Processing CreateTablet for tablet 2756beaabf064a16a8502071c2e36cae (DEFAULT_TABLE table=test-table [id=025489664ccb4221af93034a6701763c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:21.269111 8985 tablet_service.cc:1467] Processing CreateTablet for tablet 2756beaabf064a16a8502071c2e36cae (DEFAULT_TABLE table=test-table [id=025489664ccb4221af93034a6701763c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:21.269361 9125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2756beaabf064a16a8502071c2e36cae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:21.269454 8985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2756beaabf064a16a8502071c2e36cae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:21.269652 8845 tablet_service.cc:1467] Processing CreateTablet for tablet 2756beaabf064a16a8502071c2e36cae (DEFAULT_TABLE table=test-table [id=025489664ccb4221af93034a6701763c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:21.269935 8845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2756beaabf064a16a8502071c2e36cae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:21.272315 9214 tablet_bootstrap.cc:492] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379: Bootstrap starting.
I20260924 21:12:21.272679 9215 tablet_bootstrap.cc:492] T 2756beaabf064a16a8502071c2e36cae P 2fdf8641dc49447db2d5e11d735c0178: Bootstrap starting.
I20260924 21:12:21.272953 9214 tablet_bootstrap.cc:654] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:21.273325 9214 log.cc:824] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:21.273605 9215 tablet_bootstrap.cc:654] T 2756beaabf064a16a8502071c2e36cae P 2fdf8641dc49447db2d5e11d735c0178: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:21.273802 9216 tablet_bootstrap.cc:492] T 2756beaabf064a16a8502071c2e36cae P 30eebf7f3eaf46398365efa962c43c36: Bootstrap starting.
I20260924 21:12:21.273830 9215 log.cc:824] T 2756beaabf064a16a8502071c2e36cae P 2fdf8641dc49447db2d5e11d735c0178: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:21.274394 9216 tablet_bootstrap.cc:654] T 2756beaabf064a16a8502071c2e36cae P 30eebf7f3eaf46398365efa962c43c36: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:21.274585 9214 tablet_bootstrap.cc:492] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379: No bootstrap required, opened a new log
I20260924 21:12:21.274585 9215 tablet_bootstrap.cc:492] T 2756beaabf064a16a8502071c2e36cae P 2fdf8641dc49447db2d5e11d735c0178: No bootstrap required, opened a new log
I20260924 21:12:21.274652 9215 ts_tablet_manager.cc:1397] T 2756beaabf064a16a8502071c2e36cae P 2fdf8641dc49447db2d5e11d735c0178: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:12:21.274749 9216 log.cc:824] T 2756beaabf064a16a8502071c2e36cae P 30eebf7f3eaf46398365efa962c43c36: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:21.274766 9214 ts_tablet_manager.cc:1397] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:12:21.275424 9216 tablet_bootstrap.cc:492] T 2756beaabf064a16a8502071c2e36cae P 30eebf7f3eaf46398365efa962c43c36: No bootstrap required, opened a new log
I20260924 21:12:21.275522 9216 ts_tablet_manager.cc:1397] T 2756beaabf064a16a8502071c2e36cae P 30eebf7f3eaf46398365efa962c43c36: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 21:12:21.276686 9214 raft_consensus.cc:348] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } } peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } }
I20260924 21:12:21.276686 9215 raft_consensus.cc:348] T 2756beaabf064a16a8502071c2e36cae P 2fdf8641dc49447db2d5e11d735c0178 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } } peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } }
I20260924 21:12:21.276853 9214 raft_consensus.cc:374] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:21.276853 9215 raft_consensus.cc:374] T 2756beaabf064a16a8502071c2e36cae P 2fdf8641dc49447db2d5e11d735c0178 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:21.276888 9215 raft_consensus.cc:729] T 2756beaabf064a16a8502071c2e36cae P 2fdf8641dc49447db2d5e11d735c0178 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fdf8641dc49447db2d5e11d735c0178, State: Initialized, Role: FOLLOWER
I20260924 21:12:21.276888 9214 raft_consensus.cc:729] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae7d6322d6d4d3f83a208fa4d76c379, State: Initialized, Role: FOLLOWER
I20260924 21:12:21.277063 9214 consensus_queue.cc:260] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } } peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } }
I20260924 21:12:21.277127 9216 raft_consensus.cc:348] T 2756beaabf064a16a8502071c2e36cae P 30eebf7f3eaf46398365efa962c43c36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } } peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } }
I20260924 21:12:21.277297 9216 raft_consensus.cc:374] T 2756beaabf064a16a8502071c2e36cae P 30eebf7f3eaf46398365efa962c43c36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:21.277267 9215 consensus_queue.cc:260] T 2756beaabf064a16a8502071c2e36cae P 2fdf8641dc49447db2d5e11d735c0178 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } } peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } }
I20260924 21:12:21.277345 9216 raft_consensus.cc:729] T 2756beaabf064a16a8502071c2e36cae P 30eebf7f3eaf46398365efa962c43c36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30eebf7f3eaf46398365efa962c43c36, State: Initialized, Role: FOLLOWER
I20260924 21:12:21.277384 9214 ts_tablet_manager.cc:1428] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260924 21:12:21.277514 9215 ts_tablet_manager.cc:1428] T 2756beaabf064a16a8502071c2e36cae P 2fdf8641dc49447db2d5e11d735c0178: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:12:21.277489 9216 consensus_queue.cc:260] T 2756beaabf064a16a8502071c2e36cae P 30eebf7f3eaf46398365efa962c43c36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } } peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } }
I20260924 21:12:21.277549 9051 heartbeater.cc:503] Master 127.6.210.190:36861 was elected leader, sending a full tablet report...
I20260924 21:12:21.277633 9191 heartbeater.cc:503] Master 127.6.210.190:36861 was elected leader, sending a full tablet report...
I20260924 21:12:21.277760 8911 heartbeater.cc:503] Master 127.6.210.190:36861 was elected leader, sending a full tablet report...
I20260924 21:12:21.277940 9216 ts_tablet_manager.cc:1428] T 2756beaabf064a16a8502071c2e36cae P 30eebf7f3eaf46398365efa962c43c36: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
W20260924 21:12:21.325907 9052 tablet.cc:2369] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:12:21.328346 9220 raft_consensus.cc:482] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:12:21.328485 9220 raft_consensus.cc:504] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } } peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } }
I20260924 21:12:21.328902 9220 leader_election.cc:288] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2fdf8641dc49447db2d5e11d735c0178 (127.6.210.131:37881), 30eebf7f3eaf46398365efa962c43c36 (127.6.210.129:39809)
I20260924 21:12:21.331883 9212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:21.329166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:44203 (local address 127.6.210.131:37881)
0924 21:12:21.329242 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:21.329244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:21.329256 (+ 12us) server_negotiation.cc:407] Connection header received
0924 21:12:21.329287 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:21.329288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:21.329298 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:21.329362 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:21.329570 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.330151 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.331137 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.331523 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.331580 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:21.331610 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:21.331772 (+ 162us) server_negotiation.cc:299] Negotiation successful
0924 21:12:21.331810 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260924 21:12:21.331905 9213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:21.329422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:46439 (local address 127.6.210.129:39809)
0924 21:12:21.329494 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:21.329497 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:21.329506 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:12:21.329532 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:21.329533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:21.329542 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:21.329604 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:21.329753 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.330274 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.331180 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.331528 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.331621 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:21.331629 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:21.331801 (+ 172us) server_negotiation.cc:299] Negotiation successful
0924 21:12:21.331839 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260924 21:12:21.331905 8944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:21.329083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:37881 (local address 127.6.210.130:44203)
0924 21:12:21.329151 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:21.329164 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:21.329232 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:21.329397 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:21.329398 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:21.329400 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:21.329502 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:21.329505 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.330239 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.330239 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:21.331042 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:21.331045 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.331609 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.331610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:21.331689 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:21.331692 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:12:21.331743 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 21:12:21.331800 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260924 21:12:21.331931 9223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:21.329226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:39809 (local address 127.6.210.130:46439)
0924 21:12:21.329426 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:21.329436 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:21.329482 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:21.329619 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:21.329620 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:21.329621 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:21.329684 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:21.329686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.330371 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.330372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:21.331068 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:21.331070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.331632 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.331633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:21.331706 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:21.331709 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:12:21.331783 (+ 74us) client_negotiation.cc:240] Negotiation successful
0924 21:12:21.331819 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":116,"threads_started":1}
I20260924 21:12:21.332257 8865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2756beaabf064a16a8502071c2e36cae" candidate_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30eebf7f3eaf46398365efa962c43c36" is_pre_election: true
I20260924 21:12:21.332257 9145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2756beaabf064a16a8502071c2e36cae" candidate_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fdf8641dc49447db2d5e11d735c0178" is_pre_election: true
I20260924 21:12:21.332412 8865 raft_consensus.cc:2454] T 2756beaabf064a16a8502071c2e36cae P 30eebf7f3eaf46398365efa962c43c36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae7d6322d6d4d3f83a208fa4d76c379 in term 0.
I20260924 21:12:21.332412 9145 raft_consensus.cc:2454] T 2756beaabf064a16a8502071c2e36cae P 2fdf8641dc49447db2d5e11d735c0178 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae7d6322d6d4d3f83a208fa4d76c379 in term 0.
I20260924 21:12:21.332651 8938 leader_election.cc:302] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fdf8641dc49447db2d5e11d735c0178, 3ae7d6322d6d4d3f83a208fa4d76c379; no voters:
I20260924 21:12:21.332772 9220 raft_consensus.cc:2781] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:12:21.332908 9220 raft_consensus.cc:482] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:12:21.332952 9220 raft_consensus.cc:3037] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:21.333855 9220 raft_consensus.cc:504] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } } peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } }
I20260924 21:12:21.333990 9220 leader_election.cc:288] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379 [CANDIDATE]: Term 1 election: Requested vote from peers 2fdf8641dc49447db2d5e11d735c0178 (127.6.210.131:37881), 30eebf7f3eaf46398365efa962c43c36 (127.6.210.129:39809)
I20260924 21:12:21.334141 9145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2756beaabf064a16a8502071c2e36cae" candidate_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fdf8641dc49447db2d5e11d735c0178"
I20260924 21:12:21.334195 8865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2756beaabf064a16a8502071c2e36cae" candidate_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30eebf7f3eaf46398365efa962c43c36"
I20260924 21:12:21.334239 9145 raft_consensus.cc:3037] T 2756beaabf064a16a8502071c2e36cae P 2fdf8641dc49447db2d5e11d735c0178 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:21.334275 8865 raft_consensus.cc:3037] T 2756beaabf064a16a8502071c2e36cae P 30eebf7f3eaf46398365efa962c43c36 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:21.335148 8865 raft_consensus.cc:2454] T 2756beaabf064a16a8502071c2e36cae P 30eebf7f3eaf46398365efa962c43c36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae7d6322d6d4d3f83a208fa4d76c379 in term 1.
I20260924 21:12:21.335168 9145 raft_consensus.cc:2454] T 2756beaabf064a16a8502071c2e36cae P 2fdf8641dc49447db2d5e11d735c0178 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae7d6322d6d4d3f83a208fa4d76c379 in term 1.
I20260924 21:12:21.335318 8938 leader_election.cc:302] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fdf8641dc49447db2d5e11d735c0178, 3ae7d6322d6d4d3f83a208fa4d76c379; no voters:
I20260924 21:12:21.335402 9220 raft_consensus.cc:2781] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:21.335578 9220 raft_consensus.cc:686] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae7d6322d6d4d3f83a208fa4d76c379, State: Running, Role: LEADER
I20260924 21:12:21.335669 9220 consensus_queue.cc:237] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } } peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } }
I20260924 21:12:21.336522 8706 catalog_manager.cc:5495] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ae7d6322d6d4d3f83a208fa4d76c379 (127.6.210.130). New cstate: current_term: 1 leader_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } health_report { overall_health: HEALTHY } } }
Sep 24 21:12:21 dist-test-slave-q276 krb5kdc[8663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284340, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:21.362066 8776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:21.355015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47778 (local address 127.6.210.190:36861)
0924 21:12:21.355099 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:21.355101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:21.355206 (+ 105us) server_negotiation.cc:407] Connection header received
0924 21:12:21.355279 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:21.355280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:21.355290 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:21.355359 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:21.355831 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.356317 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.357038 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.357215 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.357297 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:21.359477 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:21.359478 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:21.359487 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:21.359493 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:21.360896 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:21.361290 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:21.361291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:21.361291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:21.361347 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:21.361538 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:21.361539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:21.361539 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:21.361649 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:21.361723 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:21.361871 (+ 148us) server_negotiation.cc:299] Negotiation successful
0924 21:12:21.361956 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260924 21:12:21.362141 9231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:21.354939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36861 (local address 127.0.0.1:47778)
0924 21:12:21.355120 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:21.355131 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:21.355226 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:21.355362 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:21.355363 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:21.355696 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:21.355740 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:21.355742 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.356400 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.356401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:21.356932 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:21.356933 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.357298 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.357299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:21.357352 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:21.357913 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:21.357917 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:21.359384 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:21.361000 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:21.361007 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:21.361010 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:21.361202 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:21.361411 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:21.361412 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:21.361412 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:21.361474 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:21.361712 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:21.361718 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:21.361763 (+ 45us) client_negotiation.cc:769] Sending connection context
0924 21:12:21.361851 (+ 88us) client_negotiation.cc:240] Negotiation successful
0924 21:12:21.362017 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":63,"thread_start_us":103,"threads_started":1}
I20260924 21:12:21.364281 8706 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:47778:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260924 21:12:21.364516 8706 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:12:21.365979 8845 tablet_service.cc:1467] Processing CreateTablet for tablet 6ead39df81bb4a33a14ad0976b1832e7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f2383d331c284170b06a6e3c9b838755]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 21:12:21.365993 9125 tablet_service.cc:1467] Processing CreateTablet for tablet 6ead39df81bb4a33a14ad0976b1832e7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f2383d331c284170b06a6e3c9b838755]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 21:12:21.366154 9125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ead39df81bb4a33a14ad0976b1832e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:21.366154 8845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ead39df81bb4a33a14ad0976b1832e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:21.366182 8985 tablet_service.cc:1467] Processing CreateTablet for tablet 6ead39df81bb4a33a14ad0976b1832e7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f2383d331c284170b06a6e3c9b838755]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 21:12:21.366369 8985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ead39df81bb4a33a14ad0976b1832e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:21.367455 9214 tablet_bootstrap.cc:492] T 6ead39df81bb4a33a14ad0976b1832e7 P 3ae7d6322d6d4d3f83a208fa4d76c379: Bootstrap starting.
I20260924 21:12:21.367650 9215 tablet_bootstrap.cc:492] T 6ead39df81bb4a33a14ad0976b1832e7 P 2fdf8641dc49447db2d5e11d735c0178: Bootstrap starting.
I20260924 21:12:21.367691 9216 tablet_bootstrap.cc:492] T 6ead39df81bb4a33a14ad0976b1832e7 P 30eebf7f3eaf46398365efa962c43c36: Bootstrap starting.
I20260924 21:12:21.367916 9214 tablet_bootstrap.cc:654] T 6ead39df81bb4a33a14ad0976b1832e7 P 3ae7d6322d6d4d3f83a208fa4d76c379: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:21.368201 9215 tablet_bootstrap.cc:654] T 6ead39df81bb4a33a14ad0976b1832e7 P 2fdf8641dc49447db2d5e11d735c0178: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:21.368276 9216 tablet_bootstrap.cc:654] T 6ead39df81bb4a33a14ad0976b1832e7 P 30eebf7f3eaf46398365efa962c43c36: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:21.368654 9214 tablet_bootstrap.cc:492] T 6ead39df81bb4a33a14ad0976b1832e7 P 3ae7d6322d6d4d3f83a208fa4d76c379: No bootstrap required, opened a new log
I20260924 21:12:21.368705 9214 ts_tablet_manager.cc:1397] T 6ead39df81bb4a33a14ad0976b1832e7 P 3ae7d6322d6d4d3f83a208fa4d76c379: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260924 21:12:21.368862 9215 tablet_bootstrap.cc:492] T 6ead39df81bb4a33a14ad0976b1832e7 P 2fdf8641dc49447db2d5e11d735c0178: No bootstrap required, opened a new log
I20260924 21:12:21.368906 9215 ts_tablet_manager.cc:1397] T 6ead39df81bb4a33a14ad0976b1832e7 P 2fdf8641dc49447db2d5e11d735c0178: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:21.368861 9214 raft_consensus.cc:348] T 6ead39df81bb4a33a14ad0976b1832e7 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:21.368954 9214 raft_consensus.cc:374] T 6ead39df81bb4a33a14ad0976b1832e7 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:21.368990 9214 raft_consensus.cc:729] T 6ead39df81bb4a33a14ad0976b1832e7 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae7d6322d6d4d3f83a208fa4d76c379, State: Initialized, Role: FOLLOWER
I20260924 21:12:21.369041 9214 consensus_queue.cc:260] T 6ead39df81bb4a33a14ad0976b1832e7 P 3ae7d6322d6d4d3f83a208fa4d76c379 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:21.369048 9215 raft_consensus.cc:348] T 6ead39df81bb4a33a14ad0976b1832e7 P 2fdf8641dc49447db2d5e11d735c0178 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:21.369117 9215 raft_consensus.cc:374] T 6ead39df81bb4a33a14ad0976b1832e7 P 2fdf8641dc49447db2d5e11d735c0178 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:21.369143 9215 raft_consensus.cc:729] T 6ead39df81bb4a33a14ad0976b1832e7 P 2fdf8641dc49447db2d5e11d735c0178 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fdf8641dc49447db2d5e11d735c0178, State: Initialized, Role: FOLLOWER
I20260924 21:12:21.369141 9214 ts_tablet_manager.cc:1428] T 6ead39df81bb4a33a14ad0976b1832e7 P 3ae7d6322d6d4d3f83a208fa4d76c379: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:21.369184 9215 consensus_queue.cc:260] T 6ead39df81bb4a33a14ad0976b1832e7 P 2fdf8641dc49447db2d5e11d735c0178 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:21.369194 9220 tablet_replica.cc:439] T 6ead39df81bb4a33a14ad0976b1832e7 P 3ae7d6322d6d4d3f83a208fa4d76c379: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } } }
I20260924 21:12:21.369269 9215 ts_tablet_manager.cc:1428] T 6ead39df81bb4a33a14ad0976b1832e7 P 2fdf8641dc49447db2d5e11d735c0178: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:21.369292 9220 tablet_replica.cc:442] T 6ead39df81bb4a33a14ad0976b1832e7 P 3ae7d6322d6d4d3f83a208fa4d76c379: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:12:21.369405 9221 tablet_replica.cc:439] T 6ead39df81bb4a33a14ad0976b1832e7 P 2fdf8641dc49447db2d5e11d735c0178: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } } }
I20260924 21:12:21.369513 9221 tablet_replica.cc:442] T 6ead39df81bb4a33a14ad0976b1832e7 P 2fdf8641dc49447db2d5e11d735c0178: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:12:21.369534 9216 tablet_bootstrap.cc:492] T 6ead39df81bb4a33a14ad0976b1832e7 P 30eebf7f3eaf46398365efa962c43c36: No bootstrap required, opened a new log
I20260924 21:12:21.369578 9216 ts_tablet_manager.cc:1397] T 6ead39df81bb4a33a14ad0976b1832e7 P 30eebf7f3eaf46398365efa962c43c36: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 21:12:21.369727 9216 raft_consensus.cc:348] T 6ead39df81bb4a33a14ad0976b1832e7 P 30eebf7f3eaf46398365efa962c43c36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:21.369817 9216 raft_consensus.cc:374] T 6ead39df81bb4a33a14ad0976b1832e7 P 30eebf7f3eaf46398365efa962c43c36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:21.369840 9216 raft_consensus.cc:729] T 6ead39df81bb4a33a14ad0976b1832e7 P 30eebf7f3eaf46398365efa962c43c36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30eebf7f3eaf46398365efa962c43c36, State: Initialized, Role: FOLLOWER
I20260924 21:12:21.369880 9216 consensus_queue.cc:260] T 6ead39df81bb4a33a14ad0976b1832e7 P 30eebf7f3eaf46398365efa962c43c36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:21.369973 9216 ts_tablet_manager.cc:1428] T 6ead39df81bb4a33a14ad0976b1832e7 P 30eebf7f3eaf46398365efa962c43c36: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260924 21:12:21.370036 9222 tablet_replica.cc:439] T 6ead39df81bb4a33a14ad0976b1832e7 P 30eebf7f3eaf46398365efa962c43c36: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } } }
I20260924 21:12:21.370131 9222 tablet_replica.cc:442] T 6ead39df81bb4a33a14ad0976b1832e7 P 30eebf7f3eaf46398365efa962c43c36: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:12:21.372252 9222 raft_consensus.cc:482] T 6ead39df81bb4a33a14ad0976b1832e7 P 30eebf7f3eaf46398365efa962c43c36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:12:21.372305 9222 raft_consensus.cc:504] T 6ead39df81bb4a33a14ad0976b1832e7 P 30eebf7f3eaf46398365efa962c43c36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:21.372553 9222 leader_election.cc:288] T 6ead39df81bb4a33a14ad0976b1832e7 P 30eebf7f3eaf46398365efa962c43c36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae7d6322d6d4d3f83a208fa4d76c379 (127.6.210.130:41407), 2fdf8641dc49447db2d5e11d735c0178 (127.6.210.131:37881)
I20260924 21:12:21.376477 9211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:21.372702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:58341 (local address 127.6.210.130:41407)
0924 21:12:21.372747 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:21.372750 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:21.372779 (+ 29us) server_negotiation.cc:407] Connection header received
0924 21:12:21.372853 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:21.372854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:21.372864 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:21.372913 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:21.373110 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.373837 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.375537 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.376106 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.376177 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:21.376185 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:21.376377 (+ 192us) server_negotiation.cc:299] Negotiation successful
0924 21:12:21.376412 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 21:12:21.376477 9212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:21.372880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:59043 (local address 127.6.210.131:37881)
0924 21:12:21.372940 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:21.372942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:21.372961 (+ 19us) server_negotiation.cc:407] Connection header received
0924 21:12:21.372990 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:21.372990 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:21.372998 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:21.373044 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:21.373168 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.373858 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.375537 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.376103 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.376177 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:21.376185 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:21.376378 (+ 193us) server_negotiation.cc:299] Negotiation successful
0924 21:12:21.376411 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 21:12:21.376513 8804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:21.372677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:41407 (local address 127.6.210.129:58341)
0924 21:12:21.372737 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:21.372748 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:21.372796 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:21.372956 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:21.372957 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:21.372958 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:21.373034 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:21.373036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.373975 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.373977 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:21.375430 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:21.375434 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.376173 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.376174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:21.376289 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:21.376296 (+ 7us) client_negotiation.cc:769] Sending connection context
0924 21:12:21.376362 (+ 66us) client_negotiation.cc:240] Negotiation successful
0924 21:12:21.376426 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":45}
I20260924 21:12:21.376529 9235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:21.372682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:37881 (local address 127.6.210.129:59043)
0924 21:12:21.372834 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:21.372845 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:21.372916 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:21.373044 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:21.373045 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:21.373046 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:21.373111 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:21.373113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.373990 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.373991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:21.375431 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:21.375434 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.376198 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.376199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:21.376289 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:21.376302 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 21:12:21.376362 (+ 60us) client_negotiation.cc:240] Negotiation successful
0924 21:12:21.376411 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":57,"thread_start_us":90,"threads_started":1}
I20260924 21:12:21.376794 9145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ead39df81bb4a33a14ad0976b1832e7" candidate_uuid: "30eebf7f3eaf46398365efa962c43c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fdf8641dc49447db2d5e11d735c0178" is_pre_election: true
I20260924 21:12:21.376893 9145 raft_consensus.cc:2454] T 6ead39df81bb4a33a14ad0976b1832e7 P 2fdf8641dc49447db2d5e11d735c0178 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30eebf7f3eaf46398365efa962c43c36 in term 0.
I20260924 21:12:21.376827 9005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ead39df81bb4a33a14ad0976b1832e7" candidate_uuid: "30eebf7f3eaf46398365efa962c43c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" is_pre_election: true
I20260924 21:12:21.376959 9005 raft_consensus.cc:2454] T 6ead39df81bb4a33a14ad0976b1832e7 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30eebf7f3eaf46398365efa962c43c36 in term 0.
I20260924 21:12:21.377068 8798 leader_election.cc:302] T 6ead39df81bb4a33a14ad0976b1832e7 P 30eebf7f3eaf46398365efa962c43c36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fdf8641dc49447db2d5e11d735c0178, 30eebf7f3eaf46398365efa962c43c36; no voters:
I20260924 21:12:21.377187 9222 raft_consensus.cc:2781] T 6ead39df81bb4a33a14ad0976b1832e7 P 30eebf7f3eaf46398365efa962c43c36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:12:21.377226 9222 raft_consensus.cc:482] T 6ead39df81bb4a33a14ad0976b1832e7 P 30eebf7f3eaf46398365efa962c43c36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:12:21.377255 9222 raft_consensus.cc:3037] T 6ead39df81bb4a33a14ad0976b1832e7 P 30eebf7f3eaf46398365efa962c43c36 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:21.377900 9222 raft_consensus.cc:504] T 6ead39df81bb4a33a14ad0976b1832e7 P 30eebf7f3eaf46398365efa962c43c36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:21.378047 9222 leader_election.cc:288] T 6ead39df81bb4a33a14ad0976b1832e7 P 30eebf7f3eaf46398365efa962c43c36 [CANDIDATE]: Term 1 election: Requested vote from peers 3ae7d6322d6d4d3f83a208fa4d76c379 (127.6.210.130:41407), 2fdf8641dc49447db2d5e11d735c0178 (127.6.210.131:37881)
I20260924 21:12:21.378194 9005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ead39df81bb4a33a14ad0976b1832e7" candidate_uuid: "30eebf7f3eaf46398365efa962c43c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379"
I20260924 21:12:21.378222 9145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ead39df81bb4a33a14ad0976b1832e7" candidate_uuid: "30eebf7f3eaf46398365efa962c43c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fdf8641dc49447db2d5e11d735c0178"
I20260924 21:12:21.378290 9005 raft_consensus.cc:3037] T 6ead39df81bb4a33a14ad0976b1832e7 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:21.378307 9145 raft_consensus.cc:3037] T 6ead39df81bb4a33a14ad0976b1832e7 P 2fdf8641dc49447db2d5e11d735c0178 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:21.379079 9145 raft_consensus.cc:2454] T 6ead39df81bb4a33a14ad0976b1832e7 P 2fdf8641dc49447db2d5e11d735c0178 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30eebf7f3eaf46398365efa962c43c36 in term 1.
I20260924 21:12:21.379235 9005 raft_consensus.cc:2454] T 6ead39df81bb4a33a14ad0976b1832e7 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30eebf7f3eaf46398365efa962c43c36 in term 1.
I20260924 21:12:21.379257 8798 leader_election.cc:302] T 6ead39df81bb4a33a14ad0976b1832e7 P 30eebf7f3eaf46398365efa962c43c36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fdf8641dc49447db2d5e11d735c0178, 30eebf7f3eaf46398365efa962c43c36; no voters:
I20260924 21:12:21.379345 9222 raft_consensus.cc:2781] T 6ead39df81bb4a33a14ad0976b1832e7 P 30eebf7f3eaf46398365efa962c43c36 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:21.379513 9222 raft_consensus.cc:686] T 6ead39df81bb4a33a14ad0976b1832e7 P 30eebf7f3eaf46398365efa962c43c36 [term 1 LEADER]: Becoming Leader. State: Replica: 30eebf7f3eaf46398365efa962c43c36, State: Running, Role: LEADER
I20260924 21:12:21.379613 9222 consensus_queue.cc:237] T 6ead39df81bb4a33a14ad0976b1832e7 P 30eebf7f3eaf46398365efa962c43c36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:21.380061 9236 tablet_replica.cc:439] T 6ead39df81bb4a33a14ad0976b1832e7 P 30eebf7f3eaf46398365efa962c43c36: TxnStatusTablet state changed. Reason: New leader 30eebf7f3eaf46398365efa962c43c36. Latest consensus state: current_term: 1 leader_uuid: "30eebf7f3eaf46398365efa962c43c36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } } }
I20260924 21:12:21.380149 9236 tablet_replica.cc:442] T 6ead39df81bb4a33a14ad0976b1832e7 P 30eebf7f3eaf46398365efa962c43c36: This TxnStatusTablet replica's current role is: LEADER
I20260924 21:12:21.380329 9238 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260924 21:12:21.380456 9238 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260924 21:12:21.380419 8707 catalog_manager.cc:5495] T 6ead39df81bb4a33a14ad0976b1832e7 P 30eebf7f3eaf46398365efa962c43c36 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30eebf7f3eaf46398365efa962c43c36 (127.6.210.129). New cstate: current_term: 1 leader_uuid: "30eebf7f3eaf46398365efa962c43c36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } health_report { overall_health: UNKNOWN } } }
I20260924 21:12:21.388379 9213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:21.385702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46884 (local address 127.6.210.129:39809)
0924 21:12:21.385748 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:21.385751 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:21.385762 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:12:21.385841 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:21.385842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:21.385860 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:21.385937 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:12:21.386066 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.386675 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.387496 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.387723 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.387818 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:21.388024 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:12:21.388130 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:12:21.388228 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:21.388277 (+ 49us) server_negotiation.cc:299] Negotiation successful
0924 21:12:21.388300 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 21:12:21.388412 9231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:21.385614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:39809 (local address 127.0.0.1:46884)
0924 21:12:21.385690 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:21.385703 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:21.385754 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:21.385946 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:21.385947 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:21.385948 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 21:12:21.385997 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:21.385999 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.386773 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.386774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:21.387403 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:21.387405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.387819 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.387820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:21.387891 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:21.387915 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 21:12:21.388203 (+ 288us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 21:12:21.388212 (+ 9us) client_negotiation.cc:769] Sending connection context
0924 21:12:21.388269 (+ 57us) client_negotiation.cc:240] Negotiation successful
0924 21:12:21.388320 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260924 21:12:21.390184 9145 raft_consensus.cc:1261] T 6ead39df81bb4a33a14ad0976b1832e7 P 2fdf8641dc49447db2d5e11d735c0178 [term 1 FOLLOWER]: Refusing update from remote peer 30eebf7f3eaf46398365efa962c43c36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:21.390533 9005 raft_consensus.cc:1261] T 6ead39df81bb4a33a14ad0976b1832e7 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 1 FOLLOWER]: Refusing update from remote peer 30eebf7f3eaf46398365efa962c43c36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:21.390550 9236 consensus_queue.cc:1035] T 6ead39df81bb4a33a14ad0976b1832e7 P 30eebf7f3eaf46398365efa962c43c36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:12:21.390869 9236 consensus_queue.cc:1035] T 6ead39df81bb4a33a14ad0976b1832e7 P 30eebf7f3eaf46398365efa962c43c36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:12:21.391819 9221 tablet_replica.cc:439] T 6ead39df81bb4a33a14ad0976b1832e7 P 2fdf8641dc49447db2d5e11d735c0178: TxnStatusTablet state changed. Reason: New leader 30eebf7f3eaf46398365efa962c43c36. Latest consensus state: current_term: 1 leader_uuid: "30eebf7f3eaf46398365efa962c43c36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } } }
I20260924 21:12:21.391963 9221 tablet_replica.cc:442] T 6ead39df81bb4a33a14ad0976b1832e7 P 2fdf8641dc49447db2d5e11d735c0178: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:12:21.392164 9220 tablet_replica.cc:439] T 6ead39df81bb4a33a14ad0976b1832e7 P 3ae7d6322d6d4d3f83a208fa4d76c379: TxnStatusTablet state changed. Reason: New leader 30eebf7f3eaf46398365efa962c43c36. Latest consensus state: current_term: 1 leader_uuid: "30eebf7f3eaf46398365efa962c43c36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } } }
I20260924 21:12:21.392262 9220 tablet_replica.cc:442] T 6ead39df81bb4a33a14ad0976b1832e7 P 3ae7d6322d6d4d3f83a208fa4d76c379: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:12:21.393134 9222 tablet_replica.cc:439] T 6ead39df81bb4a33a14ad0976b1832e7 P 30eebf7f3eaf46398365efa962c43c36: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "30eebf7f3eaf46398365efa962c43c36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } } }
I20260924 21:12:21.393217 9222 tablet_replica.cc:442] T 6ead39df81bb4a33a14ad0976b1832e7 P 30eebf7f3eaf46398365efa962c43c36: This TxnStatusTablet replica's current role is: LEADER
I20260924 21:12:21.393596 9222 tablet_replica.cc:439] T 6ead39df81bb4a33a14ad0976b1832e7 P 30eebf7f3eaf46398365efa962c43c36: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "30eebf7f3eaf46398365efa962c43c36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } } }
I20260924 21:12:21.393673 9222 tablet_replica.cc:442] T 6ead39df81bb4a33a14ad0976b1832e7 P 30eebf7f3eaf46398365efa962c43c36: This TxnStatusTablet replica's current role is: LEADER
I20260924 21:12:21.394250 9220 tablet_replica.cc:439] T 6ead39df81bb4a33a14ad0976b1832e7 P 3ae7d6322d6d4d3f83a208fa4d76c379: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "30eebf7f3eaf46398365efa962c43c36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } } }
I20260924 21:12:21.394353 9220 tablet_replica.cc:442] T 6ead39df81bb4a33a14ad0976b1832e7 P 3ae7d6322d6d4d3f83a208fa4d76c379: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:12:21.395092 9221 tablet_replica.cc:439] T 6ead39df81bb4a33a14ad0976b1832e7 P 2fdf8641dc49447db2d5e11d735c0178: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "30eebf7f3eaf46398365efa962c43c36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } } }
I20260924 21:12:21.395180 9221 tablet_replica.cc:442] T 6ead39df81bb4a33a14ad0976b1832e7 P 2fdf8641dc49447db2d5e11d735c0178: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:12:21.398204 9211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:21.395287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43078 (local address 127.6.210.130:41407)
0924 21:12:21.395323 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:21.395325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:21.395475 (+ 150us) server_negotiation.cc:407] Connection header received
0924 21:12:21.395514 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:21.395515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:21.395535 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:21.395596 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:12:21.395796 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.396540 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.397343 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.397644 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.397715 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:21.397845 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:12:21.397959 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:12:21.398047 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:21.398113 (+ 66us) server_negotiation.cc:299] Negotiation successful
0924 21:12:21.398136 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260924 21:12:21.398540 8912 tablet.cc:2369] T 2756beaabf064a16a8502071c2e36cae P 30eebf7f3eaf46398365efa962c43c36: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:12:21.402940 9057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:21.400434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:39809 (local address 127.6.210.130:47745)
0924 21:12:21.400504 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:21.400516 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:21.400561 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:21.400676 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:21.400677 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:21.400678 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 21:12:21.400731 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:21.400732 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.401441 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.401442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:21.402017 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:21.402019 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.402504 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.402505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:21.402574 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:21.402586 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 21:12:21.402784 (+ 198us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 21:12:21.402786 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 21:12:21.402838 (+ 52us) client_negotiation.cc:240] Negotiation successful
0924 21:12:21.402879 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260924 21:12:21.402930 9213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:21.400531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:47745 (local address 127.6.210.129:39809)
0924 21:12:21.400570 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:21.400573 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:21.400583 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:12:21.400608 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:21.400609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:21.400618 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:21.400673 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:12:21.400808 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.401366 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.402105 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.402415 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.402504 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:21.402687 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:12:21.402738 (+ 51us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:12:21.402784 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:21.402842 (+ 58us) server_negotiation.cc:299] Negotiation successful
0924 21:12:21.402862 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 21:12:21.405170 9145 raft_consensus.cc:1261] T 2756beaabf064a16a8502071c2e36cae P 2fdf8641dc49447db2d5e11d735c0178 [term 1 FOLLOWER]: Refusing update from remote peer 3ae7d6322d6d4d3f83a208fa4d76c379: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:21.405246 8865 raft_consensus.cc:1261] T 2756beaabf064a16a8502071c2e36cae P 30eebf7f3eaf46398365efa962c43c36 [term 1 FOLLOWER]: Refusing update from remote peer 3ae7d6322d6d4d3f83a208fa4d76c379: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:21.405426 9224 consensus_queue.cc:1035] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:12:21.405560 9220 consensus_queue.cc:1035] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:12:21.407086 9244 mvcc.cc:205] Tried to move back new op lower bound from 7333004662393667584 to 7333004662111318016. Current Snapshot: MvccSnapshot[applied={T|T < 7333004662393667584}]
I20260924 21:12:21.407538 9242 mvcc.cc:205] Tried to move back new op lower bound from 7333004662393667584 to 7333004662111318016. Current Snapshot: MvccSnapshot[applied={T|T < 7333004662393667584}]
I20260924 21:12:21.408268 9241 mvcc.cc:205] Tried to move back new op lower bound from 7333004662393667584 to 7333004662111318016. Current Snapshot: MvccSnapshot[applied={T|T < 7333004662393667584}]
I20260924 21:12:21.414695 9211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:21.411797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:47065 (local address 127.6.210.130:41407)
0924 21:12:21.411926 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:21.411927 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:21.411935 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:12:21.411957 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:21.411958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:21.411968 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:21.412029 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:12:21.412268 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.412742 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.413838 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.414147 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.414251 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:21.414466 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:12:21.414502 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:12:21.414564 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:21.414606 (+ 42us) server_negotiation.cc:299] Negotiation successful
0924 21:12:21.414627 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260924 21:12:21.414710 9257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:21.411659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:41407 (local address 127.6.210.129:47065)
0924 21:12:21.411777 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:21.411788 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:21.411837 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:21.412073 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:21.412074 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:21.412075 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 21:12:21.412208 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:21.412211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.412837 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.412838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:21.413757 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:21.413760 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:21.414253 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:21.414254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:21.414353 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:21.414371 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 21:12:21.414565 (+ 194us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 21:12:21.414566 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 21:12:21.414602 (+ 36us) client_negotiation.cc:240] Negotiation successful
0924 21:12:21.414640 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":46,"threads_started":1}
I20260924 21:12:21.424896 8706 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:12:21.425030 8706 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:12:21.425796 8706 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 14bb132e638b4ce3914b44d1e001055a: Sending DeleteTablet for 3 replicas of tablet 2756beaabf064a16a8502071c2e36cae
I20260924 21:12:21.426132 9125 tablet_service.cc:1514] Processing DeleteTablet for tablet 2756beaabf064a16a8502071c2e36cae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:21 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:52186
I20260924 21:12:21.426163 8845 tablet_service.cc:1514] Processing DeleteTablet for tablet 2756beaabf064a16a8502071c2e36cae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:21 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:46878
I20260924 21:12:21.426532 8985 tablet_service.cc:1514] Processing DeleteTablet for tablet 2756beaabf064a16a8502071c2e36cae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:21 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:43062
I20260924 21:12:21.426752 9263 tablet_replica.cc:330] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379: stopping tablet replica
I20260924 21:12:21.426856 9263 raft_consensus.cc:2229] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:12:21.426959 9263 raft_consensus.cc:2258] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:21.426975 8706 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47766:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:12:21.427117 8706 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:12:21.428217 9125 tablet_service.cc:1467] Processing CreateTablet for tablet 651a2a82a4284ade9f8b1a2ded60d2b8 (DEFAULT_TABLE table=test-table [id=82e9c2983bb8428a9172d0c2d6426469]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:21.428339 9125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 651a2a82a4284ade9f8b1a2ded60d2b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:21.429642 9215 tablet_bootstrap.cc:492] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 2fdf8641dc49447db2d5e11d735c0178: Bootstrap starting.
I20260924 21:12:21.430022 9215 tablet_bootstrap.cc:654] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 2fdf8641dc49447db2d5e11d735c0178: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:21.432432 8845 tablet_service.cc:1467] Processing CreateTablet for tablet 651a2a82a4284ade9f8b1a2ded60d2b8 (DEFAULT_TABLE table=test-table [id=82e9c2983bb8428a9172d0c2d6426469]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:21.432819 8845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 651a2a82a4284ade9f8b1a2ded60d2b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:21.432837 8985 tablet_service.cc:1467] Processing CreateTablet for tablet 651a2a82a4284ade9f8b1a2ded60d2b8 (DEFAULT_TABLE table=test-table [id=82e9c2983bb8428a9172d0c2d6426469]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:21.432946 8985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 651a2a82a4284ade9f8b1a2ded60d2b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:21.433727 9261 tablet_replica.cc:330] T 2756beaabf064a16a8502071c2e36cae P 2fdf8641dc49447db2d5e11d735c0178: stopping tablet replica
I20260924 21:12:21.433846 9262 tablet_replica.cc:330] T 2756beaabf064a16a8502071c2e36cae P 30eebf7f3eaf46398365efa962c43c36: stopping tablet replica
I20260924 21:12:21.434980 9216 tablet_bootstrap.cc:492] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 30eebf7f3eaf46398365efa962c43c36: Bootstrap starting.
I20260924 21:12:21.434975 9214 tablet_bootstrap.cc:492] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 3ae7d6322d6d4d3f83a208fa4d76c379: Bootstrap starting.
I20260924 21:12:21.434082 9261 raft_consensus.cc:2229] T 2756beaabf064a16a8502071c2e36cae P 2fdf8641dc49447db2d5e11d735c0178 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:12:21.435561 9262 raft_consensus.cc:2229] T 2756beaabf064a16a8502071c2e36cae P 30eebf7f3eaf46398365efa962c43c36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:12:21.435827 9216 tablet_bootstrap.cc:654] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 30eebf7f3eaf46398365efa962c43c36: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:21.435530 9214 tablet_bootstrap.cc:654] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 3ae7d6322d6d4d3f83a208fa4d76c379: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:21.436543 9263 ts_tablet_manager.cc:1905] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:21.437189 9262 raft_consensus.cc:2258] T 2756beaabf064a16a8502071c2e36cae P 30eebf7f3eaf46398365efa962c43c36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:21.435662 9261 raft_consensus.cc:2258] T 2756beaabf064a16a8502071c2e36cae P 2fdf8641dc49447db2d5e11d735c0178 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:21.438323 9263 ts_tablet_manager.cc:1918] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 21:12:21.438392 9263 log.cc:1196] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-1/wal/wals/2756beaabf064a16a8502071c2e36cae
I20260924 21:12:21.438694 9263 ts_tablet_manager.cc:1939] T 2756beaabf064a16a8502071c2e36cae P 3ae7d6322d6d4d3f83a208fa4d76c379: Deleting consensus metadata
I20260924 21:12:21.438850 9261 ts_tablet_manager.cc:1905] T 2756beaabf064a16a8502071c2e36cae P 2fdf8641dc49447db2d5e11d735c0178: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:21.439119 8695 catalog_manager.cc:4841] TS 3ae7d6322d6d4d3f83a208fa4d76c379 (127.6.210.130:41407): tablet 2756beaabf064a16a8502071c2e36cae (table test-table [id=025489664ccb4221af93034a6701763c]) successfully deleted
I20260924 21:12:21.439199 9214 tablet_bootstrap.cc:492] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 3ae7d6322d6d4d3f83a208fa4d76c379: No bootstrap required, opened a new log
I20260924 21:12:21.439244 9214 ts_tablet_manager.cc:1397] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 3ae7d6322d6d4d3f83a208fa4d76c379: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260924 21:12:21.439378 9214 raft_consensus.cc:348] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:21.439486 9214 raft_consensus.cc:374] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:21.439514 9214 raft_consensus.cc:729] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae7d6322d6d4d3f83a208fa4d76c379, State: Initialized, Role: FOLLOWER
I20260924 21:12:21.439555 9214 consensus_queue.cc:260] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 3ae7d6322d6d4d3f83a208fa4d76c379 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:21.439642 9214 ts_tablet_manager.cc:1428] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 3ae7d6322d6d4d3f83a208fa4d76c379: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260924 21:12:21.440764 9261 ts_tablet_manager.cc:1918] T 2756beaabf064a16a8502071c2e36cae P 2fdf8641dc49447db2d5e11d735c0178: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 21:12:21.440845 9261 log.cc:1196] T 2756beaabf064a16a8502071c2e36cae P 2fdf8641dc49447db2d5e11d735c0178: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-2/wal/wals/2756beaabf064a16a8502071c2e36cae
I20260924 21:12:21.441079 9261 ts_tablet_manager.cc:1939] T 2756beaabf064a16a8502071c2e36cae P 2fdf8641dc49447db2d5e11d735c0178: Deleting consensus metadata
I20260924 21:12:21.441514 9262 ts_tablet_manager.cc:1905] T 2756beaabf064a16a8502071c2e36cae P 30eebf7f3eaf46398365efa962c43c36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:21.442677 9216 tablet_bootstrap.cc:492] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 30eebf7f3eaf46398365efa962c43c36: No bootstrap required, opened a new log
I20260924 21:12:21.442659 8694 catalog_manager.cc:4841] TS 2fdf8641dc49447db2d5e11d735c0178 (127.6.210.131:37881): tablet 2756beaabf064a16a8502071c2e36cae (table test-table [id=025489664ccb4221af93034a6701763c]) successfully deleted
I20260924 21:12:21.442734 9216 ts_tablet_manager.cc:1397] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 30eebf7f3eaf46398365efa962c43c36: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20260924 21:12:21.442865 9216 raft_consensus.cc:348] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 30eebf7f3eaf46398365efa962c43c36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:21.442942 9216 raft_consensus.cc:374] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 30eebf7f3eaf46398365efa962c43c36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:21.442962 9216 raft_consensus.cc:729] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 30eebf7f3eaf46398365efa962c43c36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30eebf7f3eaf46398365efa962c43c36, State: Initialized, Role: FOLLOWER
I20260924 21:12:21.443020 9216 consensus_queue.cc:260] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 30eebf7f3eaf46398365efa962c43c36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:21.443116 9216 ts_tablet_manager.cc:1428] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 30eebf7f3eaf46398365efa962c43c36: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:21.443521 9262 ts_tablet_manager.cc:1918] T 2756beaabf064a16a8502071c2e36cae P 30eebf7f3eaf46398365efa962c43c36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 21:12:21.443588 9262 log.cc:1196] T 2756beaabf064a16a8502071c2e36cae P 30eebf7f3eaf46398365efa962c43c36: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-0/wal/wals/2756beaabf064a16a8502071c2e36cae
I20260924 21:12:21.444188 9262 ts_tablet_manager.cc:1939] T 2756beaabf064a16a8502071c2e36cae P 30eebf7f3eaf46398365efa962c43c36: Deleting consensus metadata
I20260924 21:12:21.444586 8694 catalog_manager.cc:4841] TS 30eebf7f3eaf46398365efa962c43c36 (127.6.210.129:39809): tablet 2756beaabf064a16a8502071c2e36cae (table test-table [id=025489664ccb4221af93034a6701763c]) successfully deleted
I20260924 21:12:21.450345 9215 tablet_bootstrap.cc:492] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 2fdf8641dc49447db2d5e11d735c0178: No bootstrap required, opened a new log
I20260924 21:12:21.450413 9215 ts_tablet_manager.cc:1397] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 2fdf8641dc49447db2d5e11d735c0178: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.001s
I20260924 21:12:21.450565 9215 raft_consensus.cc:348] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 2fdf8641dc49447db2d5e11d735c0178 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:21.450645 9215 raft_consensus.cc:374] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 2fdf8641dc49447db2d5e11d735c0178 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:21.450666 9215 raft_consensus.cc:729] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 2fdf8641dc49447db2d5e11d735c0178 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fdf8641dc49447db2d5e11d735c0178, State: Initialized, Role: FOLLOWER
I20260924 21:12:21.450716 9215 consensus_queue.cc:260] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 2fdf8641dc49447db2d5e11d735c0178 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:21.450809 9215 ts_tablet_manager.cc:1428] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 2fdf8641dc49447db2d5e11d735c0178: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260924 21:12:21.479039 9259 raft_consensus.cc:482] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:12:21.479177 9259 raft_consensus.cc:504] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:21.479375 9259 leader_election.cc:288] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 3ae7d6322d6d4d3f83a208fa4d76c379 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30eebf7f3eaf46398365efa962c43c36 (127.6.210.129:39809), 2fdf8641dc49447db2d5e11d735c0178 (127.6.210.131:37881)
I20260924 21:12:21.479595 8865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "651a2a82a4284ade9f8b1a2ded60d2b8" candidate_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30eebf7f3eaf46398365efa962c43c36" is_pre_election: true
I20260924 21:12:21.479691 8865 raft_consensus.cc:2454] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 30eebf7f3eaf46398365efa962c43c36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae7d6322d6d4d3f83a208fa4d76c379 in term 0.
I20260924 21:12:21.479655 9145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "651a2a82a4284ade9f8b1a2ded60d2b8" candidate_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fdf8641dc49447db2d5e11d735c0178" is_pre_election: true
I20260924 21:12:21.479756 9145 raft_consensus.cc:2454] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 2fdf8641dc49447db2d5e11d735c0178 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae7d6322d6d4d3f83a208fa4d76c379 in term 0.
I20260924 21:12:21.479894 8938 leader_election.cc:302] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 3ae7d6322d6d4d3f83a208fa4d76c379 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fdf8641dc49447db2d5e11d735c0178, 3ae7d6322d6d4d3f83a208fa4d76c379; no voters:
I20260924 21:12:21.480016 9259 raft_consensus.cc:2781] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:12:21.480059 9259 raft_consensus.cc:482] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:12:21.480085 9259 raft_consensus.cc:3037] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Advancing to term 1
W20260924 21:12:21.480695 9192 tablet.cc:2369] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 2fdf8641dc49447db2d5e11d735c0178: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:12:21.481261 9259 raft_consensus.cc:504] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:21.481617 8865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "651a2a82a4284ade9f8b1a2ded60d2b8" candidate_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30eebf7f3eaf46398365efa962c43c36"
I20260924 21:12:21.481698 8865 raft_consensus.cc:3037] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 30eebf7f3eaf46398365efa962c43c36 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:21.481709 9145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "651a2a82a4284ade9f8b1a2ded60d2b8" candidate_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fdf8641dc49447db2d5e11d735c0178"
I20260924 21:12:21.481779 9145 raft_consensus.cc:3037] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 2fdf8641dc49447db2d5e11d735c0178 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:21.482543 8865 raft_consensus.cc:2454] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 30eebf7f3eaf46398365efa962c43c36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae7d6322d6d4d3f83a208fa4d76c379 in term 1.
I20260924 21:12:21.482617 9145 raft_consensus.cc:2454] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 2fdf8641dc49447db2d5e11d735c0178 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae7d6322d6d4d3f83a208fa4d76c379 in term 1.
I20260924 21:12:21.482788 8938 leader_election.cc:302] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 3ae7d6322d6d4d3f83a208fa4d76c379 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fdf8641dc49447db2d5e11d735c0178, 3ae7d6322d6d4d3f83a208fa4d76c379; no voters:
I20260924 21:12:21.482810 9259 leader_election.cc:288] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 3ae7d6322d6d4d3f83a208fa4d76c379 [CANDIDATE]: Term 1 election: Requested vote from peers 30eebf7f3eaf46398365efa962c43c36 (127.6.210.129:39809), 2fdf8641dc49447db2d5e11d735c0178 (127.6.210.131:37881)
I20260924 21:12:21.482870 9259 raft_consensus.cc:2781] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:21.483070 9259 raft_consensus.cc:686] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae7d6322d6d4d3f83a208fa4d76c379, State: Running, Role: LEADER
I20260924 21:12:21.483135 9259 consensus_queue.cc:237] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 3ae7d6322d6d4d3f83a208fa4d76c379 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:21.483723 8706 catalog_manager.cc:5495] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 3ae7d6322d6d4d3f83a208fa4d76c379 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ae7d6322d6d4d3f83a208fa4d76c379 (127.6.210.130). New cstate: current_term: 1 leader_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } health_report { overall_health: UNKNOWN } } }
I20260924 21:12:21.499254 8865 raft_consensus.cc:1261] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 30eebf7f3eaf46398365efa962c43c36 [term 1 FOLLOWER]: Refusing update from remote peer 3ae7d6322d6d4d3f83a208fa4d76c379: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:21.499265 9145 raft_consensus.cc:1261] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 2fdf8641dc49447db2d5e11d735c0178 [term 1 FOLLOWER]: Refusing update from remote peer 3ae7d6322d6d4d3f83a208fa4d76c379: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:21.499481 9255 consensus_queue.cc:1035] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 3ae7d6322d6d4d3f83a208fa4d76c379 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:12:21.499573 9220 consensus_queue.cc:1035] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 3ae7d6322d6d4d3f83a208fa4d76c379 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 21:12:21.505314 9244 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260924 21:12:23.028728 8706 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:12:23.028828 8706 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:12:23.029668 8706 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 14bb132e638b4ce3914b44d1e001055a: Sending DeleteTablet for 3 replicas of tablet 651a2a82a4284ade9f8b1a2ded60d2b8
I20260924 21:12:23.029913 8845 tablet_service.cc:1514] Processing DeleteTablet for tablet 651a2a82a4284ade9f8b1a2ded60d2b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:23 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:46878
I20260924 21:12:23.029955 9125 tablet_service.cc:1514] Processing DeleteTablet for tablet 651a2a82a4284ade9f8b1a2ded60d2b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:23 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:52186
I20260924 21:12:23.029989 8985 tablet_service.cc:1514] Processing DeleteTablet for tablet 651a2a82a4284ade9f8b1a2ded60d2b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:23 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:43062
I20260924 21:12:23.030217 9294 tablet_replica.cc:330] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 2fdf8641dc49447db2d5e11d735c0178: stopping tablet replica
I20260924 21:12:23.030218 9293 tablet_replica.cc:330] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 3ae7d6322d6d4d3f83a208fa4d76c379: stopping tablet replica
I20260924 21:12:23.030297 9293 raft_consensus.cc:2229] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:12:23.030323 9294 raft_consensus.cc:2229] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 2fdf8641dc49447db2d5e11d735c0178 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:12:23.030386 9294 raft_consensus.cc:2258] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 2fdf8641dc49447db2d5e11d735c0178 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:23.030390 9293 raft_consensus.cc:2258] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:23.030460 9292 tablet_replica.cc:330] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 30eebf7f3eaf46398365efa962c43c36: stopping tablet replica
I20260924 21:12:23.030540 9292 raft_consensus.cc:2229] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 30eebf7f3eaf46398365efa962c43c36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:12:23.030584 9292 raft_consensus.cc:2258] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 30eebf7f3eaf46398365efa962c43c36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:23.030722 9294 ts_tablet_manager.cc:1905] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 2fdf8641dc49447db2d5e11d735c0178: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:23.030813 9292 ts_tablet_manager.cc:1905] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 30eebf7f3eaf46398365efa962c43c36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:23.032351 9292 ts_tablet_manager.cc:1918] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 30eebf7f3eaf46398365efa962c43c36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 21:12:23.032487 9292 log.cc:1196] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 30eebf7f3eaf46398365efa962c43c36: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-0/wal/wals/651a2a82a4284ade9f8b1a2ded60d2b8
I20260924 21:12:23.032611 9294 ts_tablet_manager.cc:1918] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 2fdf8641dc49447db2d5e11d735c0178: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 21:12:23.032739 9294 log.cc:1196] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 2fdf8641dc49447db2d5e11d735c0178: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-2/wal/wals/651a2a82a4284ade9f8b1a2ded60d2b8
I20260924 21:12:23.032787 9292 ts_tablet_manager.cc:1939] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 30eebf7f3eaf46398365efa962c43c36: Deleting consensus metadata
I20260924 21:12:23.032999 9294 ts_tablet_manager.cc:1939] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 2fdf8641dc49447db2d5e11d735c0178: Deleting consensus metadata
I20260924 21:12:23.033079 9293 ts_tablet_manager.cc:1905] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 3ae7d6322d6d4d3f83a208fa4d76c379: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:23.033133 8694 catalog_manager.cc:4841] TS 30eebf7f3eaf46398365efa962c43c36 (127.6.210.129:39809): tablet 651a2a82a4284ade9f8b1a2ded60d2b8 (table test-table [id=82e9c2983bb8428a9172d0c2d6426469]) successfully deleted
I20260924 21:12:23.034591 8694 catalog_manager.cc:4841] TS 2fdf8641dc49447db2d5e11d735c0178 (127.6.210.131:37881): tablet 651a2a82a4284ade9f8b1a2ded60d2b8 (table test-table [id=82e9c2983bb8428a9172d0c2d6426469]) successfully deleted
I20260924 21:12:23.034920 9293 ts_tablet_manager.cc:1918] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 3ae7d6322d6d4d3f83a208fa4d76c379: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 21:12:23.035036 9293 log.cc:1196] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 3ae7d6322d6d4d3f83a208fa4d76c379: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-1/wal/wals/651a2a82a4284ade9f8b1a2ded60d2b8
I20260924 21:12:23.035300 9293 ts_tablet_manager.cc:1939] T 651a2a82a4284ade9f8b1a2ded60d2b8 P 3ae7d6322d6d4d3f83a208fa4d76c379: Deleting consensus metadata
I20260924 21:12:23.035602 8695 catalog_manager.cc:4841] TS 3ae7d6322d6d4d3f83a208fa4d76c379 (127.6.210.130:41407): tablet 651a2a82a4284ade9f8b1a2ded60d2b8 (table test-table [id=82e9c2983bb8428a9172d0c2d6426469]) successfully deleted
Sep 24 21:12:23 dist-test-slave-q276 krb5kdc[8663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284343, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 21:12:23 dist-test-slave-q276 krb5kdc[8663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284343, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:23.061458 9302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:23.047229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47790 (local address 127.6.210.190:36861)
0924 21:12:23.047373 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:23.047375 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:23.047419 (+ 44us) server_negotiation.cc:407] Connection header received
0924 21:12:23.047541 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:23.047542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:23.047554 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:23.047640 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:23.048216 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:23.048729 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:23.049477 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:23.049682 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:23.049767 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:23.058473 (+ 8706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:23.058478 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:23.058479 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:23.058494 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:23.059831 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:23.060349 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:23.060351 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:23.060351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:23.060433 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:23.060664 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:23.060665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:23.060665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:23.060803 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:23.060917 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:23.061171 (+ 254us) server_negotiation.cc:299] Negotiation successful
0924 21:12:23.061286 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
I20260924 21:12:23.063449 8706 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47790:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:12:23.063599 8706 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:12:23.065181 8845 tablet_service.cc:1467] Processing CreateTablet for tablet 6cd6e6ab6e49436588b6bdccd1cf3344 (DEFAULT_TABLE table=test-table [id=6c4d93d8c51b49ec9e7aa35744188657]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:23.065227 8985 tablet_service.cc:1467] Processing CreateTablet for tablet 6cd6e6ab6e49436588b6bdccd1cf3344 (DEFAULT_TABLE table=test-table [id=6c4d93d8c51b49ec9e7aa35744188657]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:23.065259 9125 tablet_service.cc:1467] Processing CreateTablet for tablet 6cd6e6ab6e49436588b6bdccd1cf3344 (DEFAULT_TABLE table=test-table [id=6c4d93d8c51b49ec9e7aa35744188657]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:23.065332 8985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6cd6e6ab6e49436588b6bdccd1cf3344. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:23.065331 8845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6cd6e6ab6e49436588b6bdccd1cf3344. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:23.065383 9125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6cd6e6ab6e49436588b6bdccd1cf3344. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:23.066372 9303 tablet_bootstrap.cc:492] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 2fdf8641dc49447db2d5e11d735c0178: Bootstrap starting.
I20260924 21:12:23.066720 9305 tablet_bootstrap.cc:492] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 3ae7d6322d6d4d3f83a208fa4d76c379: Bootstrap starting.
I20260924 21:12:23.066843 9303 tablet_bootstrap.cc:654] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 2fdf8641dc49447db2d5e11d735c0178: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:23.067114 9305 tablet_bootstrap.cc:654] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 3ae7d6322d6d4d3f83a208fa4d76c379: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:23.067443 9304 tablet_bootstrap.cc:492] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 30eebf7f3eaf46398365efa962c43c36: Bootstrap starting.
I20260924 21:12:23.067515 9303 tablet_bootstrap.cc:492] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 2fdf8641dc49447db2d5e11d735c0178: No bootstrap required, opened a new log
I20260924 21:12:23.067570 9303 ts_tablet_manager.cc:1397] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 2fdf8641dc49447db2d5e11d735c0178: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:23.067700 9303 raft_consensus.cc:348] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 2fdf8641dc49447db2d5e11d735c0178 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:23.067768 9303 raft_consensus.cc:374] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 2fdf8641dc49447db2d5e11d735c0178 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:23.067782 9303 raft_consensus.cc:729] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 2fdf8641dc49447db2d5e11d735c0178 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fdf8641dc49447db2d5e11d735c0178, State: Initialized, Role: FOLLOWER
I20260924 21:12:23.067818 9304 tablet_bootstrap.cc:654] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 30eebf7f3eaf46398365efa962c43c36: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:23.067857 9303 consensus_queue.cc:260] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 2fdf8641dc49447db2d5e11d735c0178 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:23.068019 9303 ts_tablet_manager.cc:1428] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 2fdf8641dc49447db2d5e11d735c0178: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 21:12:23.068230 9305 tablet_bootstrap.cc:492] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 3ae7d6322d6d4d3f83a208fa4d76c379: No bootstrap required, opened a new log
I20260924 21:12:23.068286 9305 ts_tablet_manager.cc:1397] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 3ae7d6322d6d4d3f83a208fa4d76c379: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:12:23.068388 9305 raft_consensus.cc:348] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:23.068463 9305 raft_consensus.cc:374] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:23.068497 9305 raft_consensus.cc:729] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae7d6322d6d4d3f83a208fa4d76c379, State: Initialized, Role: FOLLOWER
I20260924 21:12:23.068548 9305 consensus_queue.cc:260] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 3ae7d6322d6d4d3f83a208fa4d76c379 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:23.068648 9305 ts_tablet_manager.cc:1428] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 3ae7d6322d6d4d3f83a208fa4d76c379: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:23.068958 9304 tablet_bootstrap.cc:492] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 30eebf7f3eaf46398365efa962c43c36: No bootstrap required, opened a new log
I20260924 21:12:23.069044 9304 ts_tablet_manager.cc:1397] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 30eebf7f3eaf46398365efa962c43c36: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:12:23.069149 9304 raft_consensus.cc:348] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 30eebf7f3eaf46398365efa962c43c36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:23.069222 9304 raft_consensus.cc:374] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 30eebf7f3eaf46398365efa962c43c36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:23.069242 9304 raft_consensus.cc:729] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 30eebf7f3eaf46398365efa962c43c36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30eebf7f3eaf46398365efa962c43c36, State: Initialized, Role: FOLLOWER
I20260924 21:12:23.069303 9304 consensus_queue.cc:260] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 30eebf7f3eaf46398365efa962c43c36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:23.069411 9304 ts_tablet_manager.cc:1428] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 30eebf7f3eaf46398365efa962c43c36: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:23.319813 9308 raft_consensus.cc:482] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 2fdf8641dc49447db2d5e11d735c0178 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:12:23.319918 9308 raft_consensus.cc:504] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 2fdf8641dc49447db2d5e11d735c0178 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:23.320217 9308 leader_election.cc:288] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 2fdf8641dc49447db2d5e11d735c0178 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30eebf7f3eaf46398365efa962c43c36 (127.6.210.129:39809), 3ae7d6322d6d4d3f83a208fa4d76c379 (127.6.210.130:41407)
I20260924 21:12:23.324051 9270 raft_consensus.cc:482] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 30eebf7f3eaf46398365efa962c43c36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:12:23.324173 9270 raft_consensus.cc:504] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 30eebf7f3eaf46398365efa962c43c36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:23.324396 9270 leader_election.cc:288] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 30eebf7f3eaf46398365efa962c43c36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ae7d6322d6d4d3f83a208fa4d76c379 (127.6.210.130:41407), 2fdf8641dc49447db2d5e11d735c0178 (127.6.210.131:37881)
I20260924 21:12:23.324579 9005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cd6e6ab6e49436588b6bdccd1cf3344" candidate_uuid: "30eebf7f3eaf46398365efa962c43c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" is_pre_election: true
I20260924 21:12:23.324575 9145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cd6e6ab6e49436588b6bdccd1cf3344" candidate_uuid: "30eebf7f3eaf46398365efa962c43c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fdf8641dc49447db2d5e11d735c0178" is_pre_election: true
I20260924 21:12:23.324683 9145 raft_consensus.cc:2454] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 2fdf8641dc49447db2d5e11d735c0178 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30eebf7f3eaf46398365efa962c43c36 in term 0.
I20260924 21:12:23.324684 9005 raft_consensus.cc:2454] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30eebf7f3eaf46398365efa962c43c36 in term 0.
I20260924 21:12:23.324730 9313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:23.320566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:47433 (local address 127.6.210.129:39809)
0924 21:12:23.320718 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:23.320720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:23.320730 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:12:23.320789 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:23.320791 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:23.320803 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:23.320884 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:23.321118 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:23.321889 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:23.323577 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:23.324178 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:23.324256 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:23.324265 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:23.324603 (+ 338us) server_negotiation.cc:299] Negotiation successful
0924 21:12:23.324656 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":90,"threads_started":1}
I20260924 21:12:23.324834 8798 leader_election.cc:302] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 30eebf7f3eaf46398365efa962c43c36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fdf8641dc49447db2d5e11d735c0178, 30eebf7f3eaf46398365efa962c43c36; no voters:
I20260924 21:12:23.324848 9310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:23.320455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:39809 (local address 127.6.210.131:47433)
0924 21:12:23.320643 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:23.320655 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:23.320716 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:23.320962 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:23.320962 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:23.320964 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:23.321021 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:23.321022 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:23.321977 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:23.321978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:23.323471 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:23.323475 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:23.324304 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:23.324305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:23.324380 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:23.324385 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 21:12:23.324437 (+ 52us) client_negotiation.cc:240] Negotiation successful
0924 21:12:23.324505 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":71,"thread_start_us":101,"threads_started":1}
I20260924 21:12:23.324981 9311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:23.320455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:41407 (local address 127.6.210.131:35737)
0924 21:12:23.320643 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:23.320655 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:23.320716 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:23.320945 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:23.320946 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:23.320948 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:23.321011 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:23.321013 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:23.321977 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:23.321978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:23.323471 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:23.323475 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:23.324326 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:23.324328 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:23.324419 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:23.324425 (+ 6us) client_negotiation.cc:769] Sending connection context
0924 21:12:23.324536 (+ 111us) client_negotiation.cc:240] Negotiation successful
0924 21:12:23.324585 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":80,"thread_start_us":116,"threads_started":1}
I20260924 21:12:23.324981 9270 raft_consensus.cc:2781] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 30eebf7f3eaf46398365efa962c43c36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:12:23.325058 9270 raft_consensus.cc:482] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 30eebf7f3eaf46398365efa962c43c36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:12:23.325084 9270 raft_consensus.cc:3037] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 30eebf7f3eaf46398365efa962c43c36 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:23.325168 8865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cd6e6ab6e49436588b6bdccd1cf3344" candidate_uuid: "2fdf8641dc49447db2d5e11d735c0178" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30eebf7f3eaf46398365efa962c43c36" is_pre_election: true
I20260924 21:12:23.325225 9312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:23.320544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:35737 (local address 127.6.210.130:41407)
0924 21:12:23.320714 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:23.320716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:23.320784 (+ 68us) server_negotiation.cc:407] Connection header received
0924 21:12:23.320808 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:23.320809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:23.320821 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:23.320887 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:23.321118 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:23.321874 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:23.323578 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:23.324179 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:23.324256 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:23.324265 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:23.324651 (+ 386us) server_negotiation.cc:299] Negotiation successful
0924 21:12:23.324834 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":107,"threads_started":1}
I20260924 21:12:23.325448 9005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cd6e6ab6e49436588b6bdccd1cf3344" candidate_uuid: "2fdf8641dc49447db2d5e11d735c0178" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" is_pre_election: true
I20260924 21:12:23.325555 9005 raft_consensus.cc:2454] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fdf8641dc49447db2d5e11d735c0178 in term 0.
I20260924 21:12:23.325721 9079 leader_election.cc:302] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 2fdf8641dc49447db2d5e11d735c0178 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fdf8641dc49447db2d5e11d735c0178, 3ae7d6322d6d4d3f83a208fa4d76c379; no voters:
I20260924 21:12:23.325948 9308 raft_consensus.cc:2781] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 2fdf8641dc49447db2d5e11d735c0178 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:12:23.325959 9270 raft_consensus.cc:504] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 30eebf7f3eaf46398365efa962c43c36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:23.326004 9308 raft_consensus.cc:482] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 2fdf8641dc49447db2d5e11d735c0178 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:12:23.326033 9308 raft_consensus.cc:3037] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 2fdf8641dc49447db2d5e11d735c0178 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:23.326099 9270 leader_election.cc:288] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 30eebf7f3eaf46398365efa962c43c36 [CANDIDATE]: Term 1 election: Requested vote from peers 3ae7d6322d6d4d3f83a208fa4d76c379 (127.6.210.130:41407), 2fdf8641dc49447db2d5e11d735c0178 (127.6.210.131:37881)
I20260924 21:12:23.326083 8865 raft_consensus.cc:2379] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 30eebf7f3eaf46398365efa962c43c36 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2fdf8641dc49447db2d5e11d735c0178 in current term 1: Already voted for candidate 30eebf7f3eaf46398365efa962c43c36 in this term.
I20260924 21:12:23.326248 9145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cd6e6ab6e49436588b6bdccd1cf3344" candidate_uuid: "30eebf7f3eaf46398365efa962c43c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fdf8641dc49447db2d5e11d735c0178"
I20260924 21:12:23.326335 9005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cd6e6ab6e49436588b6bdccd1cf3344" candidate_uuid: "30eebf7f3eaf46398365efa962c43c36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379"
I20260924 21:12:23.326385 9005 raft_consensus.cc:3037] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:23.327021 9005 raft_consensus.cc:2454] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30eebf7f3eaf46398365efa962c43c36 in term 1.
I20260924 21:12:23.327216 8799 leader_election.cc:302] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 30eebf7f3eaf46398365efa962c43c36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30eebf7f3eaf46398365efa962c43c36, 3ae7d6322d6d4d3f83a208fa4d76c379; no voters:
I20260924 21:12:23.327302 9270 raft_consensus.cc:2781] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 30eebf7f3eaf46398365efa962c43c36 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:23.327374 9270 raft_consensus.cc:686] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 30eebf7f3eaf46398365efa962c43c36 [term 1 LEADER]: Becoming Leader. State: Replica: 30eebf7f3eaf46398365efa962c43c36, State: Running, Role: LEADER
I20260924 21:12:23.327426 9270 consensus_queue.cc:237] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 30eebf7f3eaf46398365efa962c43c36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:23.327855 9308 raft_consensus.cc:504] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 2fdf8641dc49447db2d5e11d735c0178 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:23.327991 9145 raft_consensus.cc:2379] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 2fdf8641dc49447db2d5e11d735c0178 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 30eebf7f3eaf46398365efa962c43c36 in current term 1: Already voted for candidate 2fdf8641dc49447db2d5e11d735c0178 in this term.
I20260924 21:12:23.328060 8706 catalog_manager.cc:5495] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 30eebf7f3eaf46398365efa962c43c36 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30eebf7f3eaf46398365efa962c43c36 (127.6.210.129). New cstate: current_term: 1 leader_uuid: "30eebf7f3eaf46398365efa962c43c36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } health_report { overall_health: UNKNOWN } } }
I20260924 21:12:23.328357 9308 leader_election.cc:288] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 2fdf8641dc49447db2d5e11d735c0178 [CANDIDATE]: Term 1 election: Requested vote from peers 30eebf7f3eaf46398365efa962c43c36 (127.6.210.129:39809), 3ae7d6322d6d4d3f83a208fa4d76c379 (127.6.210.130:41407)
I20260924 21:12:23.328491 8865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cd6e6ab6e49436588b6bdccd1cf3344" candidate_uuid: "2fdf8641dc49447db2d5e11d735c0178" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30eebf7f3eaf46398365efa962c43c36"
I20260924 21:12:23.328634 9005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cd6e6ab6e49436588b6bdccd1cf3344" candidate_uuid: "2fdf8641dc49447db2d5e11d735c0178" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379"
I20260924 21:12:23.328716 9005 raft_consensus.cc:2379] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2fdf8641dc49447db2d5e11d735c0178 in current term 1: Already voted for candidate 30eebf7f3eaf46398365efa962c43c36 in this term.
I20260924 21:12:23.328871 9079 leader_election.cc:302] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 2fdf8641dc49447db2d5e11d735c0178 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2fdf8641dc49447db2d5e11d735c0178; no voters: 30eebf7f3eaf46398365efa962c43c36, 3ae7d6322d6d4d3f83a208fa4d76c379
I20260924 21:12:23.328954 9308 raft_consensus.cc:2726] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 2fdf8641dc49447db2d5e11d735c0178 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260924 21:12:23.353808 9313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:23.351627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46898 (local address 127.6.210.129:39809)
0924 21:12:23.351670 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:23.351672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:23.351719 (+ 47us) server_negotiation.cc:407] Connection header received
0924 21:12:23.351791 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:23.351792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:23.351801 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:23.351844 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:12:23.351944 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:23.352399 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:23.353217 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:23.353391 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:23.353439 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:23.353555 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:12:23.353578 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:12:23.353612 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:23.353735 (+ 123us) server_negotiation.cc:299] Negotiation successful
0924 21:12:23.353751 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 21:12:23.358778 9313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:23.356474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:51423 (local address 127.6.210.129:39809)
0924 21:12:23.356511 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:23.356512 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:23.356628 (+ 116us) server_negotiation.cc:407] Connection header received
0924 21:12:23.356666 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:23.356667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:23.356676 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:23.356722 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:12:23.356919 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:23.357356 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:23.358173 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:23.358362 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:23.358415 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:23.358564 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:12:23.358587 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:12:23.358624 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:23.358683 (+ 59us) server_negotiation.cc:299] Negotiation successful
0924 21:12:23.358715 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":11,"server-negotiator.queue_time_us":15}
I20260924 21:12:23.358932 9318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:23.356342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:39809 (local address 127.6.210.129:51423)
0924 21:12:23.356538 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:23.356549 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:23.356622 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:23.356764 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:23.356765 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:23.356767 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 21:12:23.356809 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:23.356810 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:23.357441 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:23.357442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:23.358095 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:23.358097 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:23.358438 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:23.358439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:23.358483 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:23.358487 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 21:12:23.358642 (+ 155us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 21:12:23.358644 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 21:12:23.358681 (+ 37us) client_negotiation.cc:240] Negotiation successful
0924 21:12:23.358727 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":2,"thread_start_us":109,"threads_started":1}
I20260924 21:12:23.360908 9145 raft_consensus.cc:1261] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 2fdf8641dc49447db2d5e11d735c0178 [term 1 FOLLOWER]: Refusing update from remote peer 30eebf7f3eaf46398365efa962c43c36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:23.360994 9005 raft_consensus.cc:1261] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 1 FOLLOWER]: Refusing update from remote peer 30eebf7f3eaf46398365efa962c43c36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:23.361112 9273 consensus_queue.cc:1035] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 30eebf7f3eaf46398365efa962c43c36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:12:23.361232 9278 consensus_queue.cc:1035] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 30eebf7f3eaf46398365efa962c43c36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:12:23.372219 8707 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:12:23.372344 8707 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:12:23.373093 8707 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 14bb132e638b4ce3914b44d1e001055a: Sending DeleteTablet for 3 replicas of tablet 6cd6e6ab6e49436588b6bdccd1cf3344
I20260924 21:12:23.373360 8985 tablet_service.cc:1514] Processing DeleteTablet for tablet 6cd6e6ab6e49436588b6bdccd1cf3344 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:23 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:43062
I20260924 21:12:23.373394 9125 tablet_service.cc:1514] Processing DeleteTablet for tablet 6cd6e6ab6e49436588b6bdccd1cf3344 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:23 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:52186
I20260924 21:12:23.373462 9293 tablet_replica.cc:330] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 3ae7d6322d6d4d3f83a208fa4d76c379: stopping tablet replica
I20260924 21:12:23.373517 9294 tablet_replica.cc:330] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 2fdf8641dc49447db2d5e11d735c0178: stopping tablet replica
I20260924 21:12:23.373545 9293 raft_consensus.cc:2229] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:12:23.373569 9294 raft_consensus.cc:2229] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 2fdf8641dc49447db2d5e11d735c0178 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:12:23.373585 9293 raft_consensus.cc:2258] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:23.373631 9294 raft_consensus.cc:2258] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 2fdf8641dc49447db2d5e11d735c0178 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:23.373705 8845 tablet_service.cc:1514] Processing DeleteTablet for tablet 6cd6e6ab6e49436588b6bdccd1cf3344 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:23 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:46878
I20260924 21:12:23.373793 9292 tablet_replica.cc:330] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 30eebf7f3eaf46398365efa962c43c36: stopping tablet replica
I20260924 21:12:23.373853 9292 raft_consensus.cc:2229] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 30eebf7f3eaf46398365efa962c43c36 [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:12:23.373935 9292 raft_consensus.cc:2258] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 30eebf7f3eaf46398365efa962c43c36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:23.374048 9294 ts_tablet_manager.cc:1905] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 2fdf8641dc49447db2d5e11d735c0178: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:23.374204 9292 ts_tablet_manager.cc:1905] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 30eebf7f3eaf46398365efa962c43c36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:23.374478 9293 ts_tablet_manager.cc:1905] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 3ae7d6322d6d4d3f83a208fa4d76c379: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:23.375808 9294 ts_tablet_manager.cc:1918] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 2fdf8641dc49447db2d5e11d735c0178: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 21:12:23.375867 9294 log.cc:1196] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 2fdf8641dc49447db2d5e11d735c0178: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-2/wal/wals/6cd6e6ab6e49436588b6bdccd1cf3344
I20260924 21:12:23.375931 9292 ts_tablet_manager.cc:1918] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 30eebf7f3eaf46398365efa962c43c36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 21:12:23.375974 9292 log.cc:1196] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 30eebf7f3eaf46398365efa962c43c36: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-0/wal/wals/6cd6e6ab6e49436588b6bdccd1cf3344
I20260924 21:12:23.375988 9293 ts_tablet_manager.cc:1918] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 3ae7d6322d6d4d3f83a208fa4d76c379: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 21:12:23.376050 9293 log.cc:1196] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 3ae7d6322d6d4d3f83a208fa4d76c379: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-1/wal/wals/6cd6e6ab6e49436588b6bdccd1cf3344
I20260924 21:12:23.376153 9294 ts_tablet_manager.cc:1939] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 2fdf8641dc49447db2d5e11d735c0178: Deleting consensus metadata
I20260924 21:12:23.376174 9292 ts_tablet_manager.cc:1939] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 30eebf7f3eaf46398365efa962c43c36: Deleting consensus metadata
I20260924 21:12:23.376263 9293 ts_tablet_manager.cc:1939] T 6cd6e6ab6e49436588b6bdccd1cf3344 P 3ae7d6322d6d4d3f83a208fa4d76c379: Deleting consensus metadata
I20260924 21:12:23.376464 8694 catalog_manager.cc:4841] TS 2fdf8641dc49447db2d5e11d735c0178 (127.6.210.131:37881): tablet 6cd6e6ab6e49436588b6bdccd1cf3344 (table test-table [id=6c4d93d8c51b49ec9e7aa35744188657]) successfully deleted
I20260924 21:12:23.376433 8707 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47790:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:12:23.376595 8707 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:12:23.376857 8694 catalog_manager.cc:4841] TS 30eebf7f3eaf46398365efa962c43c36 (127.6.210.129:39809): tablet 6cd6e6ab6e49436588b6bdccd1cf3344 (table test-table [id=6c4d93d8c51b49ec9e7aa35744188657]) successfully deleted
I20260924 21:12:23.376988 8695 catalog_manager.cc:4841] TS 3ae7d6322d6d4d3f83a208fa4d76c379 (127.6.210.130:41407): tablet 6cd6e6ab6e49436588b6bdccd1cf3344 (table test-table [id=6c4d93d8c51b49ec9e7aa35744188657]) successfully deleted
I20260924 21:12:23.377702 8985 tablet_service.cc:1467] Processing CreateTablet for tablet 8441c3217aa44eabbe5544330b262979 (DEFAULT_TABLE table=test-table [id=0a64342d715b4643b6a0057042e07fa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:23.377745 9125 tablet_service.cc:1467] Processing CreateTablet for tablet 8441c3217aa44eabbe5544330b262979 (DEFAULT_TABLE table=test-table [id=0a64342d715b4643b6a0057042e07fa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:23.377817 8985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8441c3217aa44eabbe5544330b262979. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:23.377821 9125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8441c3217aa44eabbe5544330b262979. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:23.379020 9303 tablet_bootstrap.cc:492] T 8441c3217aa44eabbe5544330b262979 P 2fdf8641dc49447db2d5e11d735c0178: Bootstrap starting.
I20260924 21:12:23.379372 9303 tablet_bootstrap.cc:654] T 8441c3217aa44eabbe5544330b262979 P 2fdf8641dc49447db2d5e11d735c0178: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:23.379733 9305 tablet_bootstrap.cc:492] T 8441c3217aa44eabbe5544330b262979 P 3ae7d6322d6d4d3f83a208fa4d76c379: Bootstrap starting.
I20260924 21:12:23.380025 9305 tablet_bootstrap.cc:654] T 8441c3217aa44eabbe5544330b262979 P 3ae7d6322d6d4d3f83a208fa4d76c379: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:23.380409 8845 tablet_service.cc:1467] Processing CreateTablet for tablet 8441c3217aa44eabbe5544330b262979 (DEFAULT_TABLE table=test-table [id=0a64342d715b4643b6a0057042e07fa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:23.380518 8845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8441c3217aa44eabbe5544330b262979. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:23.381387 9304 tablet_bootstrap.cc:492] T 8441c3217aa44eabbe5544330b262979 P 30eebf7f3eaf46398365efa962c43c36: Bootstrap starting.
I20260924 21:12:23.381531 9305 tablet_bootstrap.cc:492] T 8441c3217aa44eabbe5544330b262979 P 3ae7d6322d6d4d3f83a208fa4d76c379: No bootstrap required, opened a new log
I20260924 21:12:23.381577 9305 ts_tablet_manager.cc:1397] T 8441c3217aa44eabbe5544330b262979 P 3ae7d6322d6d4d3f83a208fa4d76c379: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:12:23.381659 9305 raft_consensus.cc:348] T 8441c3217aa44eabbe5544330b262979 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:23.381705 9305 raft_consensus.cc:374] T 8441c3217aa44eabbe5544330b262979 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:23.381718 9305 raft_consensus.cc:729] T 8441c3217aa44eabbe5544330b262979 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ae7d6322d6d4d3f83a208fa4d76c379, State: Initialized, Role: FOLLOWER
I20260924 21:12:23.381752 9304 tablet_bootstrap.cc:654] T 8441c3217aa44eabbe5544330b262979 P 30eebf7f3eaf46398365efa962c43c36: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:23.381779 9305 consensus_queue.cc:260] T 8441c3217aa44eabbe5544330b262979 P 3ae7d6322d6d4d3f83a208fa4d76c379 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:23.381891 9305 ts_tablet_manager.cc:1428] T 8441c3217aa44eabbe5544330b262979 P 3ae7d6322d6d4d3f83a208fa4d76c379: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 21:12:23.382180 9303 tablet_bootstrap.cc:492] T 8441c3217aa44eabbe5544330b262979 P 2fdf8641dc49447db2d5e11d735c0178: No bootstrap required, opened a new log
I20260924 21:12:23.382234 9303 ts_tablet_manager.cc:1397] T 8441c3217aa44eabbe5544330b262979 P 2fdf8641dc49447db2d5e11d735c0178: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260924 21:12:23.382369 9304 tablet_bootstrap.cc:492] T 8441c3217aa44eabbe5544330b262979 P 30eebf7f3eaf46398365efa962c43c36: No bootstrap required, opened a new log
I20260924 21:12:23.382427 9304 ts_tablet_manager.cc:1397] T 8441c3217aa44eabbe5544330b262979 P 30eebf7f3eaf46398365efa962c43c36: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:23.382438 9303 raft_consensus.cc:348] T 8441c3217aa44eabbe5544330b262979 P 2fdf8641dc49447db2d5e11d735c0178 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:23.382519 9303 raft_consensus.cc:374] T 8441c3217aa44eabbe5544330b262979 P 2fdf8641dc49447db2d5e11d735c0178 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:23.382546 9303 raft_consensus.cc:729] T 8441c3217aa44eabbe5544330b262979 P 2fdf8641dc49447db2d5e11d735c0178 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fdf8641dc49447db2d5e11d735c0178, State: Initialized, Role: FOLLOWER
I20260924 21:12:23.382529 9304 raft_consensus.cc:348] T 8441c3217aa44eabbe5544330b262979 P 30eebf7f3eaf46398365efa962c43c36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:23.382601 9304 raft_consensus.cc:374] T 8441c3217aa44eabbe5544330b262979 P 30eebf7f3eaf46398365efa962c43c36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:23.382594 9303 consensus_queue.cc:260] T 8441c3217aa44eabbe5544330b262979 P 2fdf8641dc49447db2d5e11d735c0178 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:23.382629 9304 raft_consensus.cc:729] T 8441c3217aa44eabbe5544330b262979 P 30eebf7f3eaf46398365efa962c43c36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30eebf7f3eaf46398365efa962c43c36, State: Initialized, Role: FOLLOWER
I20260924 21:12:23.382678 9304 consensus_queue.cc:260] T 8441c3217aa44eabbe5544330b262979 P 30eebf7f3eaf46398365efa962c43c36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:23.382699 9303 ts_tablet_manager.cc:1428] T 8441c3217aa44eabbe5544330b262979 P 2fdf8641dc49447db2d5e11d735c0178: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:23.382772 9304 ts_tablet_manager.cc:1428] T 8441c3217aa44eabbe5544330b262979 P 30eebf7f3eaf46398365efa962c43c36: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:23.642823 9285 raft_consensus.cc:482] T 8441c3217aa44eabbe5544330b262979 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:12:23.642925 9285 raft_consensus.cc:504] T 8441c3217aa44eabbe5544330b262979 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:23.643083 9285 leader_election.cc:288] T 8441c3217aa44eabbe5544330b262979 P 3ae7d6322d6d4d3f83a208fa4d76c379 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30eebf7f3eaf46398365efa962c43c36 (127.6.210.129:39809), 2fdf8641dc49447db2d5e11d735c0178 (127.6.210.131:37881)
I20260924 21:12:23.643316 8865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8441c3217aa44eabbe5544330b262979" candidate_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30eebf7f3eaf46398365efa962c43c36" is_pre_election: true
I20260924 21:12:23.643319 9145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8441c3217aa44eabbe5544330b262979" candidate_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fdf8641dc49447db2d5e11d735c0178" is_pre_election: true
I20260924 21:12:23.643416 9145 raft_consensus.cc:2454] T 8441c3217aa44eabbe5544330b262979 P 2fdf8641dc49447db2d5e11d735c0178 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae7d6322d6d4d3f83a208fa4d76c379 in term 0.
I20260924 21:12:23.643416 8865 raft_consensus.cc:2454] T 8441c3217aa44eabbe5544330b262979 P 30eebf7f3eaf46398365efa962c43c36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ae7d6322d6d4d3f83a208fa4d76c379 in term 0.
I20260924 21:12:23.643633 8938 leader_election.cc:302] T 8441c3217aa44eabbe5544330b262979 P 3ae7d6322d6d4d3f83a208fa4d76c379 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fdf8641dc49447db2d5e11d735c0178, 3ae7d6322d6d4d3f83a208fa4d76c379; no voters:
I20260924 21:12:23.643744 9285 raft_consensus.cc:2781] T 8441c3217aa44eabbe5544330b262979 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:12:23.643787 9285 raft_consensus.cc:482] T 8441c3217aa44eabbe5544330b262979 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:12:23.643811 9285 raft_consensus.cc:3037] T 8441c3217aa44eabbe5544330b262979 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:23.644505 9285 raft_consensus.cc:504] T 8441c3217aa44eabbe5544330b262979 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:23.644629 9285 leader_election.cc:288] T 8441c3217aa44eabbe5544330b262979 P 3ae7d6322d6d4d3f83a208fa4d76c379 [CANDIDATE]: Term 1 election: Requested vote from peers 30eebf7f3eaf46398365efa962c43c36 (127.6.210.129:39809), 2fdf8641dc49447db2d5e11d735c0178 (127.6.210.131:37881)
I20260924 21:12:23.644819 8865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8441c3217aa44eabbe5544330b262979" candidate_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30eebf7f3eaf46398365efa962c43c36"
I20260924 21:12:23.644819 9145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8441c3217aa44eabbe5544330b262979" candidate_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fdf8641dc49447db2d5e11d735c0178"
I20260924 21:12:23.644904 9145 raft_consensus.cc:3037] T 8441c3217aa44eabbe5544330b262979 P 2fdf8641dc49447db2d5e11d735c0178 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:23.644904 8865 raft_consensus.cc:3037] T 8441c3217aa44eabbe5544330b262979 P 30eebf7f3eaf46398365efa962c43c36 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:23.645625 8865 raft_consensus.cc:2454] T 8441c3217aa44eabbe5544330b262979 P 30eebf7f3eaf46398365efa962c43c36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae7d6322d6d4d3f83a208fa4d76c379 in term 1.
I20260924 21:12:23.645627 9145 raft_consensus.cc:2454] T 8441c3217aa44eabbe5544330b262979 P 2fdf8641dc49447db2d5e11d735c0178 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ae7d6322d6d4d3f83a208fa4d76c379 in term 1.
I20260924 21:12:23.645812 8938 leader_election.cc:302] T 8441c3217aa44eabbe5544330b262979 P 3ae7d6322d6d4d3f83a208fa4d76c379 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fdf8641dc49447db2d5e11d735c0178, 3ae7d6322d6d4d3f83a208fa4d76c379; no voters:
I20260924 21:12:23.645928 9285 raft_consensus.cc:2781] T 8441c3217aa44eabbe5544330b262979 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:23.646040 9285 raft_consensus.cc:686] T 8441c3217aa44eabbe5544330b262979 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 1 LEADER]: Becoming Leader. State: Replica: 3ae7d6322d6d4d3f83a208fa4d76c379, State: Running, Role: LEADER
I20260924 21:12:23.646083 9285 consensus_queue.cc:237] T 8441c3217aa44eabbe5544330b262979 P 3ae7d6322d6d4d3f83a208fa4d76c379 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } }
I20260924 21:12:23.646744 8707 catalog_manager.cc:5495] T 8441c3217aa44eabbe5544330b262979 P 3ae7d6322d6d4d3f83a208fa4d76c379 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ae7d6322d6d4d3f83a208fa4d76c379 (127.6.210.130). New cstate: current_term: 1 leader_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ae7d6322d6d4d3f83a208fa4d76c379" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 41407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 } health_report { overall_health: UNKNOWN } } }
I20260924 21:12:23.671056 9312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:23.668445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43090 (local address 127.6.210.130:41407)
0924 21:12:23.668520 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:23.668522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:23.668533 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:12:23.668598 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:23.668599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:23.668611 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:23.668666 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:12:23.668809 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:23.669348 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:23.670166 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:23.670428 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:23.670514 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:23.670704 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:12:23.670753 (+ 49us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:12:23.670873 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:23.670938 (+ 65us) server_negotiation.cc:299] Negotiation successful
0924 21:12:23.670963 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260924 21:12:23.673170 9144 raft_consensus.cc:1261] T 8441c3217aa44eabbe5544330b262979 P 2fdf8641dc49447db2d5e11d735c0178 [term 1 FOLLOWER]: Refusing update from remote peer 3ae7d6322d6d4d3f83a208fa4d76c379: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:23.673202 8865 raft_consensus.cc:1261] T 8441c3217aa44eabbe5544330b262979 P 30eebf7f3eaf46398365efa962c43c36 [term 1 FOLLOWER]: Refusing update from remote peer 3ae7d6322d6d4d3f83a208fa4d76c379: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:23.673488 9325 consensus_queue.cc:1035] T 8441c3217aa44eabbe5544330b262979 P 3ae7d6322d6d4d3f83a208fa4d76c379 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fdf8641dc49447db2d5e11d735c0178" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:12:23.673615 9285 consensus_queue.cc:1035] T 8441c3217aa44eabbe5544330b262979 P 3ae7d6322d6d4d3f83a208fa4d76c379 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30eebf7f3eaf46398365efa962c43c36" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 21:12:23.678107 9275 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260924 21:12:25.196019 8708 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:12:25.196117 8708 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:12:25.196947 8708 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 14bb132e638b4ce3914b44d1e001055a: Sending DeleteTablet for 3 replicas of tablet 8441c3217aa44eabbe5544330b262979
I20260924 21:12:25.197206 8985 tablet_service.cc:1514] Processing DeleteTablet for tablet 8441c3217aa44eabbe5544330b262979 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:25 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:43062
I20260924 21:12:25.197263 8845 tablet_service.cc:1514] Processing DeleteTablet for tablet 8441c3217aa44eabbe5544330b262979 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:25 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:46878
I20260924 21:12:25.197327 9125 tablet_service.cc:1514] Processing DeleteTablet for tablet 8441c3217aa44eabbe5544330b262979 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:25 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:52186
I20260924 21:12:25.197398 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 8780
I20260924 21:12:25.198887 9354 tablet_replica.cc:330] T 8441c3217aa44eabbe5544330b262979 P 2fdf8641dc49447db2d5e11d735c0178: stopping tablet replica
I20260924 21:12:25.198974 9354 raft_consensus.cc:2229] T 8441c3217aa44eabbe5544330b262979 P 2fdf8641dc49447db2d5e11d735c0178 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:12:25.199031 9354 raft_consensus.cc:2258] T 8441c3217aa44eabbe5544330b262979 P 2fdf8641dc49447db2d5e11d735c0178 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:25.199149 9352 tablet_replica.cc:330] T 8441c3217aa44eabbe5544330b262979 P 3ae7d6322d6d4d3f83a208fa4d76c379: stopping tablet replica
I20260924 21:12:25.199220 9352 raft_consensus.cc:2229] T 8441c3217aa44eabbe5544330b262979 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:12:25.199299 9352 raft_consensus.cc:2258] T 8441c3217aa44eabbe5544330b262979 P 3ae7d6322d6d4d3f83a208fa4d76c379 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:25.199302 9354 ts_tablet_manager.cc:1905] T 8441c3217aa44eabbe5544330b262979 P 2fdf8641dc49447db2d5e11d735c0178: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:25.199571 9352 ts_tablet_manager.cc:1905] T 8441c3217aa44eabbe5544330b262979 P 3ae7d6322d6d4d3f83a208fa4d76c379: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:25.200989 9354 ts_tablet_manager.cc:1918] T 8441c3217aa44eabbe5544330b262979 P 2fdf8641dc49447db2d5e11d735c0178: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 21:12:25.201056 9354 log.cc:1196] T 8441c3217aa44eabbe5544330b262979 P 2fdf8641dc49447db2d5e11d735c0178: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-2/wal/wals/8441c3217aa44eabbe5544330b262979
I20260924 21:12:25.201301 9354 ts_tablet_manager.cc:1939] T 8441c3217aa44eabbe5544330b262979 P 2fdf8641dc49447db2d5e11d735c0178: Deleting consensus metadata
I20260924 21:12:25.201633 8694 catalog_manager.cc:4841] TS 2fdf8641dc49447db2d5e11d735c0178 (127.6.210.131:37881): tablet 8441c3217aa44eabbe5544330b262979 (table test-table [id=0a64342d715b4643b6a0057042e07fa3]) successfully deleted
I20260924 21:12:25.201711 9352 ts_tablet_manager.cc:1918] T 8441c3217aa44eabbe5544330b262979 P 3ae7d6322d6d4d3f83a208fa4d76c379: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 21:12:25.201787 9352 log.cc:1196] T 8441c3217aa44eabbe5544330b262979 P 3ae7d6322d6d4d3f83a208fa4d76c379: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284333163253-6986-0/minicluster-data/ts-1/wal/wals/8441c3217aa44eabbe5544330b262979
I20260924 21:12:25.202066 9352 ts_tablet_manager.cc:1939] T 8441c3217aa44eabbe5544330b262979 P 3ae7d6322d6d4d3f83a208fa4d76c379: Deleting consensus metadata
I20260924 21:12:25.202430 8695 catalog_manager.cc:4841] TS 3ae7d6322d6d4d3f83a208fa4d76c379 (127.6.210.130:41407): tablet 8441c3217aa44eabbe5544330b262979 (table test-table [id=0a64342d715b4643b6a0057042e07fa3]) successfully deleted
W20260924 21:12:25.205703 8694 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.210.129:39809 (error 108)
I20260924 21:12:25.205983 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 8920
I20260924 21:12:25.206770 9356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:25.206152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:39809 (local address 127.0.0.1:46906)
0924 21:12:25.206665 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:25.206720 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.129:39809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":71,"threads_started":1}
W20260924 21:12:25.206924 8694 catalog_manager.cc:4584] TS 30eebf7f3eaf46398365efa962c43c36 (127.6.210.129:39809): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8441c3217aa44eabbe5544330b262979: Network error: Client connection negotiation failed: client connection to 127.6.210.129:39809: connect: Connection refused (error 111)
I20260924 21:12:25.212179 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 9060
I20260924 21:12:25.217767 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 8676
2026-09-24T21:12:25Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5815 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:12:25 dist-test-slave-q276 krb5kdc[9359](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:12:25 dist-test-slave-q276 krb5kdc[9359](info): set up 2 sockets
Sep 24 21:12:25 dist-test-slave-q276 krb5kdc[9359](info): commencing operation
krb5kdc: starting...
W20260924 21:12:26.359078 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:12:26 dist-test-slave-q276 krb5kdc[9359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:12:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:12:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.210.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
I20260924 21:12:26.477444 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:35261
--webserver_interface=127.6.210.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:39721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:35261
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:26.557327 9372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:26.557531 9372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:26.557554 9372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:26.558736 9372 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:12:26.558779 9372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:26.558794 9372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:26.558805 9372 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:12:26.558817 9372 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:12:26.560225 9372 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:39721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:35261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:35261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:26.560411 9372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:26.560575 9372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:26.565867 9378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:26.565907 9372 server_base.cc:1034] running on GCE node
W20260924 21:12:26.565830 9380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:26.565874 9377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:26.566375 9372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:26.566705 9372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:26.567859 9372 hybrid_clock.cc:648] HybridClock initialized: now 1790284346567848 us; error 27 us; skew 500 ppm
Sep 24 21:12:26 dist-test-slave-q276 krb5kdc[9359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284346, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:26.570528 9372 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:12:26.570942 9372 webserver.cc:460] Webserver started at http://127.6.210.190:41627/ using document root <none> and password file <none>
I20260924 21:12:26.571148 9372 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:26.571188 9372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:26.571281 9372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:26.572325 9372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "8cff78e13b594dc588e6bb7e803023e9"
format_stamp: "Formatted at 2026-09-24 21:12:26 on dist-test-slave-q276"
server_key: "554f0ba3ebbbbb54eace6b1a63084724"
server_key_iv: "8da7024387a34bc08c58788e03e43900"
server_key_version: "encryptionkey@0"
I20260924 21:12:26.572641 9372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "8cff78e13b594dc588e6bb7e803023e9"
format_stamp: "Formatted at 2026-09-24 21:12:26 on dist-test-slave-q276"
server_key: "554f0ba3ebbbbb54eace6b1a63084724"
server_key_iv: "8da7024387a34bc08c58788e03e43900"
server_key_version: "encryptionkey@0"
I20260924 21:12:26.573899 9372 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:12:26.574611 9387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:26.574805 9372 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:26.574880 9372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "8cff78e13b594dc588e6bb7e803023e9"
format_stamp: "Formatted at 2026-09-24 21:12:26 on dist-test-slave-q276"
server_key: "554f0ba3ebbbbb54eace6b1a63084724"
server_key_iv: "8da7024387a34bc08c58788e03e43900"
server_key_version: "encryptionkey@0"
I20260924 21:12:26.574978 9372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:26.581811 9372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:26.584070 9372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:26.584224 9372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:26.588282 9372 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:35261
I20260924 21:12:26.588315 9439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:35261 every 8 connection(s)
I20260924 21:12:26.588663 9372 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:12:26.589512 9440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:26.591881 9440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cff78e13b594dc588e6bb7e803023e9: Bootstrap starting.
I20260924 21:12:26.591949 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 9372
I20260924 21:12:26.592095 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:12:26.592276 6986 external_mini_cluster.cc:1428] Setting key 7f652189c191917ec0e4413049226d0e
I20260924 21:12:26.592582 9440 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cff78e13b594dc588e6bb7e803023e9: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:26.592913 9440 log.cc:824] T 00000000000000000000000000000000 P 8cff78e13b594dc588e6bb7e803023e9: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:26.593583 9440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cff78e13b594dc588e6bb7e803023e9: No bootstrap required, opened a new log
I20260924 21:12:26.595439 9440 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cff78e13b594dc588e6bb7e803023e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cff78e13b594dc588e6bb7e803023e9" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 35261 } }
I20260924 21:12:26.595592 9440 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8cff78e13b594dc588e6bb7e803023e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:26.595638 9440 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cff78e13b594dc588e6bb7e803023e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cff78e13b594dc588e6bb7e803023e9, State: Initialized, Role: FOLLOWER
I20260924 21:12:26.595805 9440 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cff78e13b594dc588e6bb7e803023e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cff78e13b594dc588e6bb7e803023e9" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 35261 } }
I20260924 21:12:26.595901 9440 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cff78e13b594dc588e6bb7e803023e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:12:26.595943 9440 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cff78e13b594dc588e6bb7e803023e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:12:26.595991 9440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cff78e13b594dc588e6bb7e803023e9 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:26.596654 9440 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cff78e13b594dc588e6bb7e803023e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cff78e13b594dc588e6bb7e803023e9" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 35261 } }
I20260924 21:12:26.596761 9440 leader_election.cc:302] T 00000000000000000000000000000000 P 8cff78e13b594dc588e6bb7e803023e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cff78e13b594dc588e6bb7e803023e9; no voters:
I20260924 21:12:26.596927 9440 leader_election.cc:288] T 00000000000000000000000000000000 P 8cff78e13b594dc588e6bb7e803023e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:12:26.596968 9445 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cff78e13b594dc588e6bb7e803023e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:26.597154 9445 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cff78e13b594dc588e6bb7e803023e9 [term 1 LEADER]: Becoming Leader. State: Replica: 8cff78e13b594dc588e6bb7e803023e9, State: Running, Role: LEADER
I20260924 21:12:26.597182 9440 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cff78e13b594dc588e6bb7e803023e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:12:26.597298 9445 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cff78e13b594dc588e6bb7e803023e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cff78e13b594dc588e6bb7e803023e9" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 35261 } }
Sep 24 21:12:26 dist-test-slave-q276 krb5kdc[9359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:26.599347 9447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cff78e13b594dc588e6bb7e803023e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cff78e13b594dc588e6bb7e803023e9. Latest consensus state: current_term: 1 leader_uuid: "8cff78e13b594dc588e6bb7e803023e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cff78e13b594dc588e6bb7e803023e9" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 35261 } } }
I20260924 21:12:26.599423 9447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cff78e13b594dc588e6bb7e803023e9 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:26.599882 9461 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:12:26.599725 9446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cff78e13b594dc588e6bb7e803023e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8cff78e13b594dc588e6bb7e803023e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cff78e13b594dc588e6bb7e803023e9" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 35261 } } }
I20260924 21:12:26.599977 9446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cff78e13b594dc588e6bb7e803023e9 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:26.600394 9461 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:12:26.601897 9461 catalog_manager.cc:1347] Generated new cluster ID: 06e39b4a350d41b987821da91b42e417
I20260924 21:12:26.601946 9461 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:12:26.603657 9443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:26.592767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55004 (local address 127.6.210.190:35261)
0924 21:12:26.592952 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:26.592968 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:26.592992 (+ 24us) server_negotiation.cc:407] Connection header received
0924 21:12:26.593418 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:26.593426 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:26.593468 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:26.593630 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:26.594276 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:26.595161 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:26.596056 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:26.596718 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:26.596837 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:26.599979 (+ 3142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:26.599983 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:26.599984 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:26.599996 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:26.602035 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:26.602437 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:26.602440 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:26.602440 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:26.602548 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:26.602752 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:26.602753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:26.602754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:26.603061 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:26.603217 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:26.603305 (+ 88us) server_negotiation.cc:299] Negotiation successful
0924 21:12:26.603439 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":75,"threads_started":1}
I20260924 21:12:26.619868 9461 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:12:26.620606 9461 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:12:26.631771 9461 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8cff78e13b594dc588e6bb7e803023e9: Generated new TSK 0
I20260924 21:12:26.631994 9461 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.210.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
I20260924 21:12:26.668659 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.129:0
--local_ip_for_outbound_sockets=127.6.210.129
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:35261
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:39721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:26.750551 9467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:26.750763 9467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:26.750787 9467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:26.751883 9467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:26.751915 9467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:26.751941 9467 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.129
I20260924 21:12:26.753281 9467 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:39721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:35261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.129
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:26.753475 9467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:26.753665 9467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:26.758533 9472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:26.758543 9475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:26.758622 9473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:26.758643 9467 server_base.cc:1034] running on GCE node
I20260924 21:12:26.758981 9467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:26.759182 9467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:26.760345 9467 hybrid_clock.cc:648] HybridClock initialized: now 1790284346760332 us; error 44 us; skew 500 ppm
Sep 24 21:12:26 dist-test-slave-q276 krb5kdc[9359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284346, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:26.762957 9467 init.cc:377] Logged in from keytab as kudu/127.6.210.129@KRBTEST.COM (short username kudu)
I20260924 21:12:26.763434 9467 webserver.cc:460] Webserver started at http://127.6.210.129:39663/ using document root <none> and password file <none>
I20260924 21:12:26.763664 9467 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:26.763731 9467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:26.763881 9467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:26.764951 9467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/data/instance:
uuid: "3ab8d8109f61409c8c09f88c7133f004"
format_stamp: "Formatted at 2026-09-24 21:12:26 on dist-test-slave-q276"
server_key: "9eebfc3568d3292b26e0f14507060862"
server_key_iv: "e94e6ec8f088ea35d865e3e2276725ea"
server_key_version: "encryptionkey@0"
I20260924 21:12:26.765323 9467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance:
uuid: "3ab8d8109f61409c8c09f88c7133f004"
format_stamp: "Formatted at 2026-09-24 21:12:26 on dist-test-slave-q276"
server_key: "9eebfc3568d3292b26e0f14507060862"
server_key_iv: "e94e6ec8f088ea35d865e3e2276725ea"
server_key_version: "encryptionkey@0"
I20260924 21:12:26.766719 9467 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:12:26.767416 9482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:26.767580 9467 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:12:26.767679 9467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/wal
uuid: "3ab8d8109f61409c8c09f88c7133f004"
format_stamp: "Formatted at 2026-09-24 21:12:26 on dist-test-slave-q276"
server_key: "9eebfc3568d3292b26e0f14507060862"
server_key_iv: "e94e6ec8f088ea35d865e3e2276725ea"
server_key_version: "encryptionkey@0"
I20260924 21:12:26.767756 9467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:26.794250 9467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:26.796731 9467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:26.796900 9467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:26.797142 9467 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:26.797483 9467 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:26.797565 9467 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:26.797608 9467 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:26.797636 9467 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:26.803682 9467 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.129:33469
I20260924 21:12:26.803722 9595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.129:33469 every 8 connection(s)
I20260924 21:12:26.804049 9467 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
Sep 24 21:12:26 dist-test-slave-q276 krb5kdc[9359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284346, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:26.813467 9443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:26.804746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:59091 (local address 127.6.210.190:35261)
0924 21:12:26.804792 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:26.804795 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:26.805078 (+ 283us) server_negotiation.cc:407] Connection header received
0924 21:12:26.805540 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:26.805541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:26.805550 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:26.805609 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:26.806695 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:26.807196 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:26.808268 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:26.808486 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:26.808545 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:26.810813 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:26.810814 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:26.810814 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:26.810821 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:26.812175 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:26.812589 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:26.812590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:26.812591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:26.812648 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:26.812871 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:26.812872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:26.812872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:26.813025 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:26.813079 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:26.813327 (+ 248us) server_negotiation.cc:299] Negotiation successful
0924 21:12:26.813399 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 21:12:26.813634 9598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:26.804725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:35261 (local address 127.6.210.129:59091)
0924 21:12:26.804891 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:26.804933 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:26.805447 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:26.805691 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:26.805694 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:26.806380 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:26.806595 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:26.806600 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:26.807304 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:26.807305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:26.808175 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:26.808177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:26.808572 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:26.808573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:26.808679 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:26.809197 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:26.809206 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:26.810706 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:26.812250 (+ 1544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:26.812255 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:26.812257 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:26.812466 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:26.812714 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:26.812714 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:26.812715 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:26.812813 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:26.813081 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:26.813085 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:26.813198 (+ 113us) client_negotiation.cc:769] Sending connection context
0924 21:12:26.813278 (+ 80us) client_negotiation.cc:240] Negotiation successful
0924 21:12:26.813397 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":94,"threads_started":1}
I20260924 21:12:26.813732 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 9467
I20260924 21:12:26.813810 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance
I20260924 21:12:26.814004 6986 external_mini_cluster.cc:1428] Setting key b4c1d61f42f903010ccadb6f2d2c2248
I20260924 21:12:26.814213 9596 heartbeater.cc:347] Connected to a master server at 127.6.210.190:35261
I20260924 21:12:26.814339 9596 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:26.814556 9596 heartbeater.cc:511] Master 127.6.210.190:35261 requested a full tablet report, sending...
I20260924 21:12:26.815111 9404 ts_manager.cc:194] Registered new tserver with Master: 3ab8d8109f61409c8c09f88c7133f004 (127.6.210.129:33469)
I20260924 21:12:26.816054 9404 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.129@KRBTEST.COM'} at 127.6.210.129:59091
WARNING: no policy specified for kudu/127.6.210.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
I20260924 21:12:26.838555 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.130:0
--local_ip_for_outbound_sockets=127.6.210.130
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:35261
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:39721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:26.924356 9601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:26.924579 9601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:26.924605 9601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:26.925820 9601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:26.925865 9601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:26.925892 9601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.130
I20260924 21:12:26.927424 9601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:39721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:35261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.130
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:26.927687 9601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:26.927925 9601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:26.933319 9607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:26.933365 9606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:26.933354 9609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:26.933421 9601 server_base.cc:1034] running on GCE node
I20260924 21:12:26.933753 9601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:26.934006 9601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:26.935192 9601 hybrid_clock.cc:648] HybridClock initialized: now 1790284346935177 us; error 31 us; skew 500 ppm
Sep 24 21:12:26 dist-test-slave-q276 krb5kdc[9359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284346, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:26.937809 9601 init.cc:377] Logged in from keytab as kudu/127.6.210.130@KRBTEST.COM (short username kudu)
I20260924 21:12:26.938238 9601 webserver.cc:460] Webserver started at http://127.6.210.130:42155/ using document root <none> and password file <none>
I20260924 21:12:26.938532 9601 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:26.938603 9601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:26.938746 9601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:26.939826 9601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/data/instance:
uuid: "28e607cfb09141b29f1f78c7362c17d4"
format_stamp: "Formatted at 2026-09-24 21:12:26 on dist-test-slave-q276"
server_key: "e84dc0f737bd37ede54157529fdaa3b3"
server_key_iv: "ae0b31a9c9374489e5d4a0686d904fcd"
server_key_version: "encryptionkey@0"
I20260924 21:12:26.940189 9601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance:
uuid: "28e607cfb09141b29f1f78c7362c17d4"
format_stamp: "Formatted at 2026-09-24 21:12:26 on dist-test-slave-q276"
server_key: "e84dc0f737bd37ede54157529fdaa3b3"
server_key_iv: "ae0b31a9c9374489e5d4a0686d904fcd"
server_key_version: "encryptionkey@0"
I20260924 21:12:26.941493 9601 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:12:26.942232 9616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:26.942431 9601 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:12:26.942545 9601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/wal
uuid: "28e607cfb09141b29f1f78c7362c17d4"
format_stamp: "Formatted at 2026-09-24 21:12:26 on dist-test-slave-q276"
server_key: "e84dc0f737bd37ede54157529fdaa3b3"
server_key_iv: "ae0b31a9c9374489e5d4a0686d904fcd"
server_key_version: "encryptionkey@0"
I20260924 21:12:26.942618 9601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:26.957305 9601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:26.959755 9601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:26.959919 9601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:26.960156 9601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:26.960461 9601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:26.960513 9601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:26.960566 9601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:26.960589 9601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:26.966455 9601 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.130:40915
I20260924 21:12:26.966532 9729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.130:40915 every 8 connection(s)
I20260924 21:12:26.966885 9601 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
Sep 24 21:12:26 dist-test-slave-q276 krb5kdc[9359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284346, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:26.974439 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 9601
I20260924 21:12:26.974551 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance
I20260924 21:12:26.974740 6986 external_mini_cluster.cc:1428] Setting key c267eadd1d971dc7cf6b7d78b5f08999
I20260924 21:12:26.977108 9443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:26.967724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:40731 (local address 127.6.210.190:35261)
0924 21:12:26.967800 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:26.967803 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:26.967954 (+ 151us) server_negotiation.cc:407] Connection header received
0924 21:12:26.968407 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:26.968408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:26.968418 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:26.968470 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:26.969371 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:26.969856 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:26.970898 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:26.971094 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:26.971156 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:26.973753 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:26.973755 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:26.973755 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:26.973762 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:26.975579 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:26.976016 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:26.976017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:26.976017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:26.976083 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:26.976370 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:26.976371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:26.976372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:26.976497 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:26.976569 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:26.976872 (+ 303us) server_negotiation.cc:299] Negotiation successful
0924 21:12:26.976958 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260924 21:12:26.977272 9732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:26.967677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:35261 (local address 127.6.210.130:40731)
0924 21:12:26.967840 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:26.967876 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:26.968330 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:26.968534 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:26.968537 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:26.969089 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:26.969282 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:26.969287 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:26.969931 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:26.969932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:26.970827 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:26.970830 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:26.971223 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:26.971224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:26.971425 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:26.972040 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:26.972049 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:26.973635 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:26.975668 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:26.975677 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:26.975681 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:26.975931 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:26.976173 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:26.976174 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:26.976174 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:26.976299 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:26.976570 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:26.976578 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:26.976738 (+ 160us) client_negotiation.cc:769] Sending connection context
0924 21:12:26.976851 (+ 113us) client_negotiation.cc:240] Negotiation successful
0924 21:12:26.977021 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":101,"threads_started":1}
I20260924 21:12:26.978112 9730 heartbeater.cc:347] Connected to a master server at 127.6.210.190:35261
I20260924 21:12:26.978221 9730 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:26.978451 9730 heartbeater.cc:511] Master 127.6.210.190:35261 requested a full tablet report, sending...
I20260924 21:12:26.978912 9404 ts_manager.cc:194] Registered new tserver with Master: 28e607cfb09141b29f1f78c7362c17d4 (127.6.210.130:40915)
I20260924 21:12:26.979321 9404 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.130@KRBTEST.COM'} at 127.6.210.130:40731
WARNING: no policy specified for kudu/127.6.210.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
I20260924 21:12:27.001864 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.131:0
--local_ip_for_outbound_sockets=127.6.210.131
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:35261
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:39721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:27.088835 9735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:27.089177 9735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:27.089212 9735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:27.090476 9735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:27.090518 9735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:27.090544 9735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.131
I20260924 21:12:27.091996 9735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:39721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:35261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.131
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:27.092212 9735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:27.092392 9735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:27.098049 9740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:27.098131 9741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:27.098332 9735 server_base.cc:1034] running on GCE node
W20260924 21:12:27.098049 9743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:27.098762 9735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:27.099081 9735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:27.100271 9735 hybrid_clock.cc:648] HybridClock initialized: now 1790284347100249 us; error 39 us; skew 500 ppm
Sep 24 21:12:27 dist-test-slave-q276 krb5kdc[9359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284347, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:27.103026 9735 init.cc:377] Logged in from keytab as kudu/127.6.210.131@KRBTEST.COM (short username kudu)
I20260924 21:12:27.103511 9735 webserver.cc:460] Webserver started at http://127.6.210.131:35419/ using document root <none> and password file <none>
I20260924 21:12:27.103781 9735 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:27.103863 9735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:27.104050 9735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:27.105235 9735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/data/instance:
uuid: "8802e14f1a60473d9a4dc5b59e33b8e0"
format_stamp: "Formatted at 2026-09-24 21:12:27 on dist-test-slave-q276"
server_key: "c7d38628aff482502de5e5413c9939c0"
server_key_iv: "cc5eeb2f15706f83f597a9ad4721a038"
server_key_version: "encryptionkey@0"
I20260924 21:12:27.105636 9735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance:
uuid: "8802e14f1a60473d9a4dc5b59e33b8e0"
format_stamp: "Formatted at 2026-09-24 21:12:27 on dist-test-slave-q276"
server_key: "c7d38628aff482502de5e5413c9939c0"
server_key_iv: "cc5eeb2f15706f83f597a9ad4721a038"
server_key_version: "encryptionkey@0"
I20260924 21:12:27.107128 9735 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:12:27.107919 9750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:27.108165 9735 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:27.108274 9735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/wal
uuid: "8802e14f1a60473d9a4dc5b59e33b8e0"
format_stamp: "Formatted at 2026-09-24 21:12:27 on dist-test-slave-q276"
server_key: "c7d38628aff482502de5e5413c9939c0"
server_key_iv: "cc5eeb2f15706f83f597a9ad4721a038"
server_key_version: "encryptionkey@0"
I20260924 21:12:27.108354 9735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:27.126693 9735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:27.128652 9735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:27.128822 9735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:27.129067 9735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:27.129400 9735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:27.129455 9735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:27.129499 9735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:27.129523 9735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:27.135679 9735 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.131:34303
I20260924 21:12:27.135725 9863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.131:34303 every 8 connection(s)
I20260924 21:12:27.136061 9735 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
I20260924 21:12:27.137132 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 9735
I20260924 21:12:27.137234 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance
I20260924 21:12:27.137399 6986 external_mini_cluster.cc:1428] Setting key edf9ac0285dea87a07cfcf6b16b313ea
Sep 24 21:12:27 dist-test-slave-q276 krb5kdc[9359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284347, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:27.145510 9443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:27.136755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:53271 (local address 127.6.210.190:35261)
0924 21:12:27.136800 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:27.136803 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:27.137031 (+ 228us) server_negotiation.cc:407] Connection header received
0924 21:12:27.137731 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:27.137732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:27.137744 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:27.137803 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:27.138849 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:27.139345 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:27.140348 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:27.140540 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:27.140631 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:27.142822 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:27.142824 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:27.142825 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:27.142832 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:27.144102 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:27.144492 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:27.144493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:27.144493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:27.144552 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:27.144797 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:27.144798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:27.144798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:27.144918 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:27.144993 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:27.145331 (+ 338us) server_negotiation.cc:299] Negotiation successful
0924 21:12:27.145422 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 21:12:27.145702 9866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:27.136737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:35261 (local address 127.6.210.131:53271)
0924 21:12:27.136884 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:27.136927 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:27.137616 (+ 689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:27.137871 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:27.137875 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:27.138533 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:27.138746 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:27.138751 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:27.139435 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:27.139436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:27.140243 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:27.140245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:27.140650 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:27.140651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:27.140783 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:27.141316 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:27.141324 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:27.142727 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:27.144182 (+ 1455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:27.144189 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:27.144191 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:27.144406 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:27.144615 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:27.144617 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:27.144617 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:27.144728 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:27.144983 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:27.144989 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:27.145182 (+ 193us) client_negotiation.cc:769] Sending connection context
0924 21:12:27.145310 (+ 128us) client_negotiation.cc:240] Negotiation successful
0924 21:12:27.145460 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":65,"threads_started":1}
I20260924 21:12:27.146184 9864 heartbeater.cc:347] Connected to a master server at 127.6.210.190:35261
I20260924 21:12:27.146306 9864 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:27.146486 9864 heartbeater.cc:511] Master 127.6.210.190:35261 requested a full tablet report, sending...
I20260924 21:12:27.146867 9404 ts_manager.cc:194] Registered new tserver with Master: 8802e14f1a60473d9a4dc5b59e33b8e0 (127.6.210.131:34303)
I20260924 21:12:27.147199 9404 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.131@KRBTEST.COM'} at 127.6.210.131:53271
I20260924 21:12:27.149127 6986 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260924 21:12:27.161309 9443 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:27.160376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55006 (local address 127.6.210.190:35261)
0924 21:12:27.160467 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:27.160470 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:27.160547 (+ 77us) server_negotiation.cc:407] Connection header received
0924 21:12:27.160596 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:27.160597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:27.160606 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:27.160658 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:27.161238 (+ 580us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55006: BlockingRecv error: recv got EOF from 127.0.0.1:55006 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260924 21:12:27.162361 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 9467
I20260924 21:12:27.166474 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 9601
I20260924 21:12:27.170735 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 9735
I20260924 21:12:27.174703 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 9372
2026-09-24T21:12:27Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1955 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:12:27 dist-test-slave-q276 krb5kdc[9876](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:12:27 dist-test-slave-q276 krb5kdc[9876](info): set up 2 sockets
Sep 24 21:12:27 dist-test-slave-q276 krb5kdc[9876](info): commencing operation
krb5kdc: starting...
W20260924 21:12:28.231681 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:12:28 dist-test-slave-q276 krb5kdc[9876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:12:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:12:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.210.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
I20260924 21:12:28.349000 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:34007
--webserver_interface=127.6.210.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:34597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:34007
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:28.435392 9889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:28.435611 9889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:28.435634 9889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:28.436759 9889 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:12:28.436796 9889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:28.436810 9889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:28.436823 9889 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:12:28.436836 9889 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:12:28.438249 9889 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:34597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:34007
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:34007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:28.438459 9889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:28.438633 9889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:28.443643 9895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:28.443652 9894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:28.443697 9889 server_base.cc:1034] running on GCE node
W20260924 21:12:28.443650 9897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:28.444058 9889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:28.444355 9889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:28.445521 9889 hybrid_clock.cc:648] HybridClock initialized: now 1790284348445516 us; error 36 us; skew 500 ppm
Sep 24 21:12:28 dist-test-slave-q276 krb5kdc[9876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284348, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:28.453964 9889 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:12:28.454484 9889 webserver.cc:460] Webserver started at http://127.6.210.190:35547/ using document root <none> and password file <none>
I20260924 21:12:28.454680 9889 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:28.454722 9889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:28.454818 9889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:28.455924 9889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "b7e931cfa6724a21a514d7148bf78aab"
format_stamp: "Formatted at 2026-09-24 21:12:28 on dist-test-slave-q276"
server_key: "b9d7aceba526a18d999f64a02dc3c12c"
server_key_iv: "5e3ee9b19156eeb2d8fa3a9503830b69"
server_key_version: "encryptionkey@0"
I20260924 21:12:28.456243 9889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "b7e931cfa6724a21a514d7148bf78aab"
format_stamp: "Formatted at 2026-09-24 21:12:28 on dist-test-slave-q276"
server_key: "b9d7aceba526a18d999f64a02dc3c12c"
server_key_iv: "5e3ee9b19156eeb2d8fa3a9503830b69"
server_key_version: "encryptionkey@0"
I20260924 21:12:28.457691 9889 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:12:28.458567 9904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:28.458789 9889 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:28.458897 9889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "b7e931cfa6724a21a514d7148bf78aab"
format_stamp: "Formatted at 2026-09-24 21:12:28 on dist-test-slave-q276"
server_key: "b9d7aceba526a18d999f64a02dc3c12c"
server_key_iv: "5e3ee9b19156eeb2d8fa3a9503830b69"
server_key_version: "encryptionkey@0"
I20260924 21:12:28.458984 9889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:28.469563 9889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:28.472092 9889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:28.472208 9889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:28.476213 9889 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:34007
I20260924 21:12:28.476294 9956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:34007 every 8 connection(s)
I20260924 21:12:28.476650 9889 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:12:28.477596 9957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:28.480094 9957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7e931cfa6724a21a514d7148bf78aab: Bootstrap starting.
I20260924 21:12:28.480719 9957 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7e931cfa6724a21a514d7148bf78aab: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:28.481040 9957 log.cc:824] T 00000000000000000000000000000000 P b7e931cfa6724a21a514d7148bf78aab: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:28.481796 9957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7e931cfa6724a21a514d7148bf78aab: No bootstrap required, opened a new log
I20260924 21:12:28.483500 9957 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7e931cfa6724a21a514d7148bf78aab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e931cfa6724a21a514d7148bf78aab" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34007 } }
I20260924 21:12:28.483660 9957 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7e931cfa6724a21a514d7148bf78aab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:28.483705 9957 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7e931cfa6724a21a514d7148bf78aab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7e931cfa6724a21a514d7148bf78aab, State: Initialized, Role: FOLLOWER
I20260924 21:12:28.483894 9957 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7e931cfa6724a21a514d7148bf78aab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e931cfa6724a21a514d7148bf78aab" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34007 } }
I20260924 21:12:28.483994 9957 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7e931cfa6724a21a514d7148bf78aab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:12:28.483970 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 9889
I20260924 21:12:28.484041 9957 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7e931cfa6724a21a514d7148bf78aab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:12:28.484086 9957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7e931cfa6724a21a514d7148bf78aab [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:28.484103 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:12:28.484272 6986 external_mini_cluster.cc:1428] Setting key 93fd86c18f0c8ba7b3b54e8a07e9eb06
I20260924 21:12:28.485221 9957 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7e931cfa6724a21a514d7148bf78aab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e931cfa6724a21a514d7148bf78aab" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34007 } }
I20260924 21:12:28.485329 9957 leader_election.cc:302] T 00000000000000000000000000000000 P b7e931cfa6724a21a514d7148bf78aab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7e931cfa6724a21a514d7148bf78aab; no voters:
I20260924 21:12:28.485487 9957 leader_election.cc:288] T 00000000000000000000000000000000 P b7e931cfa6724a21a514d7148bf78aab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:12:28.485545 9962 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7e931cfa6724a21a514d7148bf78aab [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:28.485745 9962 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7e931cfa6724a21a514d7148bf78aab [term 1 LEADER]: Becoming Leader. State: Replica: b7e931cfa6724a21a514d7148bf78aab, State: Running, Role: LEADER
I20260924 21:12:28.485764 9957 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7e931cfa6724a21a514d7148bf78aab [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:12:28.486539 9962 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7e931cfa6724a21a514d7148bf78aab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e931cfa6724a21a514d7148bf78aab" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34007 } }
I20260924 21:12:28.487121 9963 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7e931cfa6724a21a514d7148bf78aab [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7e931cfa6724a21a514d7148bf78aab. Latest consensus state: current_term: 1 leader_uuid: "b7e931cfa6724a21a514d7148bf78aab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e931cfa6724a21a514d7148bf78aab" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34007 } } }
I20260924 21:12:28.487218 9963 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7e931cfa6724a21a514d7148bf78aab [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:28.487646 9964 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7e931cfa6724a21a514d7148bf78aab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7e931cfa6724a21a514d7148bf78aab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7e931cfa6724a21a514d7148bf78aab" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34007 } } }
I20260924 21:12:28.487737 9977 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:12:28.488368 9977 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:12:28.488888 9964 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7e931cfa6724a21a514d7148bf78aab [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:28.490713 9977 catalog_manager.cc:1347] Generated new cluster ID: 3d514881066d445fad6fcf1c3e02ddd5
I20260924 21:12:28.490764 9977 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 21:12:28 dist-test-slave-q276 krb5kdc[9876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:28.496580 9961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:28.484806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52362 (local address 127.6.210.190:34007)
0924 21:12:28.485038 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:28.485047 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:28.485083 (+ 36us) server_negotiation.cc:407] Connection header received
0924 21:12:28.485200 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:28.485208 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:28.485266 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:28.485427 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:28.486329 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:28.487781 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:28.488870 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:28.489475 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:28.489603 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:28.492478 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:28.492481 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:28.492482 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:28.492494 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:28.494520 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:28.494938 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:28.494942 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:28.494942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:28.495429 (+ 487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:28.495674 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:28.495675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:28.495675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:28.495970 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:28.496115 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:28.496250 (+ 135us) server_negotiation.cc:299] Negotiation successful
0924 21:12:28.496388 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":124,"threads_started":1}
I20260924 21:12:28.497606 6986 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:12:28.507634 9977 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:12:28.508198 9977 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:12:28.513401 9977 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7e931cfa6724a21a514d7148bf78aab: Generated new TSK 0
I20260924 21:12:28.513598 9977 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:12:28.517144 9961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:28.513207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52364 (local address 127.6.210.190:34007)
0924 21:12:28.513243 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:28.513245 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:28.513440 (+ 195us) server_negotiation.cc:407] Connection header received
0924 21:12:28.513552 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:28.513553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:28.513564 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:28.513593 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:28.514075 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:28.514643 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:28.515407 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:28.515625 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:28.515710 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:28.516527 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:28.516528 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:28.516529 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:12:28.516739 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:28.516756 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:12:28.516764 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:28.516833 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:28.517017 (+ 184us) server_negotiation.cc:299] Negotiation successful
0924 21:12:28.517045 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260924 21:12:28.519843 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 9889
2026-09-24T21:12:28Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1343 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:12:28 dist-test-slave-q276 krb5kdc[9990](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:12:28 dist-test-slave-q276 krb5kdc[9990](info): set up 2 sockets
Sep 24 21:12:28 dist-test-slave-q276 krb5kdc[9990](info): commencing operation
krb5kdc: starting...
W20260924 21:12:29.567215 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:12:29 dist-test-slave-q276 krb5kdc[9990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:12:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:12:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.210.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
I20260924 21:12:29.685364 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:44511
--webserver_interface=127.6.210.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:34515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:44511
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:29.764947 10003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:29.765202 10003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:29.765241 10003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:29.766449 10003 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:12:29.766522 10003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:29.766547 10003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:29.766577 10003 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:12:29.766640 10003 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:12:29.768090 10003 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:34515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:44511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:44511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:29.768368 10003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:29.768589 10003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:29.773561 10011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:29.773545 10008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:29.773550 10009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:29.773797 10003 server_base.cc:1034] running on GCE node
I20260924 21:12:29.774099 10003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:29.774427 10003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:29.775606 10003 hybrid_clock.cc:648] HybridClock initialized: now 1790284349775586 us; error 48 us; skew 500 ppm
Sep 24 21:12:29 dist-test-slave-q276 krb5kdc[9990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284349, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:29.778324 10003 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:12:29.778746 10003 webserver.cc:460] Webserver started at http://127.6.210.190:35281/ using document root <none> and password file <none>
I20260924 21:12:29.778977 10003 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:29.779044 10003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:29.779189 10003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:29.780287 10003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "a5ff97fb05e548e5bb1b2d70a66c4419"
format_stamp: "Formatted at 2026-09-24 21:12:29 on dist-test-slave-q276"
server_key: "fddd525a6b94b39c50816e812666237c"
server_key_iv: "8b6d4399879d32585d57494ca1d49c8e"
server_key_version: "encryptionkey@0"
I20260924 21:12:29.780648 10003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "a5ff97fb05e548e5bb1b2d70a66c4419"
format_stamp: "Formatted at 2026-09-24 21:12:29 on dist-test-slave-q276"
server_key: "fddd525a6b94b39c50816e812666237c"
server_key_iv: "8b6d4399879d32585d57494ca1d49c8e"
server_key_version: "encryptionkey@0"
I20260924 21:12:29.781988 10003 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:12:29.782779 10018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:29.782976 10003 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 21:12:29.783075 10003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "a5ff97fb05e548e5bb1b2d70a66c4419"
format_stamp: "Formatted at 2026-09-24 21:12:29 on dist-test-slave-q276"
server_key: "fddd525a6b94b39c50816e812666237c"
server_key_iv: "8b6d4399879d32585d57494ca1d49c8e"
server_key_version: "encryptionkey@0"
I20260924 21:12:29.783151 10003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:29.798936 10003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:29.800798 10003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:29.801028 10003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:29.805414 10003 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:44511
I20260924 21:12:29.805454 10070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:44511 every 8 connection(s)
I20260924 21:12:29.805853 10003 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:12:29.806947 10071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:29.809154 10071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5ff97fb05e548e5bb1b2d70a66c4419: Bootstrap starting.
I20260924 21:12:29.809731 10071 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5ff97fb05e548e5bb1b2d70a66c4419: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:29.810014 10071 log.cc:824] T 00000000000000000000000000000000 P a5ff97fb05e548e5bb1b2d70a66c4419: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:29.810290 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 10003
I20260924 21:12:29.810405 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:12:29.810595 6986 external_mini_cluster.cc:1428] Setting key d7f7787041be99b67aab44ab0c4c0956
I20260924 21:12:29.810850 10071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5ff97fb05e548e5bb1b2d70a66c4419: No bootstrap required, opened a new log
I20260924 21:12:29.812561 10071 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5ff97fb05e548e5bb1b2d70a66c4419 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5ff97fb05e548e5bb1b2d70a66c4419" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 44511 } }
I20260924 21:12:29.812829 10071 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5ff97fb05e548e5bb1b2d70a66c4419 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:29.812882 10071 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5ff97fb05e548e5bb1b2d70a66c4419 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5ff97fb05e548e5bb1b2d70a66c4419, State: Initialized, Role: FOLLOWER
I20260924 21:12:29.813045 10071 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5ff97fb05e548e5bb1b2d70a66c4419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5ff97fb05e548e5bb1b2d70a66c4419" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 44511 } }
I20260924 21:12:29.813146 10071 raft_consensus.cc:388] T 00000000000000000000000000000000 P a5ff97fb05e548e5bb1b2d70a66c4419 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:12:29.813205 10071 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5ff97fb05e548e5bb1b2d70a66c4419 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:12:29.813280 10071 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5ff97fb05e548e5bb1b2d70a66c4419 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:29.814182 10071 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5ff97fb05e548e5bb1b2d70a66c4419 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5ff97fb05e548e5bb1b2d70a66c4419" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 44511 } }
I20260924 21:12:29.814323 10071 leader_election.cc:302] T 00000000000000000000000000000000 P a5ff97fb05e548e5bb1b2d70a66c4419 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a5ff97fb05e548e5bb1b2d70a66c4419; no voters:
I20260924 21:12:29.814517 10071 leader_election.cc:288] T 00000000000000000000000000000000 P a5ff97fb05e548e5bb1b2d70a66c4419 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:12:29.814535 10076 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5ff97fb05e548e5bb1b2d70a66c4419 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:29.814647 10076 raft_consensus.cc:686] T 00000000000000000000000000000000 P a5ff97fb05e548e5bb1b2d70a66c4419 [term 1 LEADER]: Becoming Leader. State: Replica: a5ff97fb05e548e5bb1b2d70a66c4419, State: Running, Role: LEADER
I20260924 21:12:29.814779 10071 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5ff97fb05e548e5bb1b2d70a66c4419 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:12:29.814779 10076 consensus_queue.cc:237] T 00000000000000000000000000000000 P a5ff97fb05e548e5bb1b2d70a66c4419 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5ff97fb05e548e5bb1b2d70a66c4419" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 44511 } }
I20260924 21:12:29.815821 10078 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5ff97fb05e548e5bb1b2d70a66c4419 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5ff97fb05e548e5bb1b2d70a66c4419. Latest consensus state: current_term: 1 leader_uuid: "a5ff97fb05e548e5bb1b2d70a66c4419" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5ff97fb05e548e5bb1b2d70a66c4419" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 44511 } } }
I20260924 21:12:29.815900 10078 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5ff97fb05e548e5bb1b2d70a66c4419 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:29.816018 10077 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5ff97fb05e548e5bb1b2d70a66c4419 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a5ff97fb05e548e5bb1b2d70a66c4419" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5ff97fb05e548e5bb1b2d70a66c4419" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 44511 } } }
I20260924 21:12:29.816123 10077 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5ff97fb05e548e5bb1b2d70a66c4419 [sys.catalog]: This master's current role is: LEADER
W20260924 21:12:29.816668 10092 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a5ff97fb05e548e5bb1b2d70a66c4419: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 21:12:29.816763 10092 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 21:12:29.816848 10088 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:12:29.817368 10088 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 24 21:12:29 dist-test-slave-q276 krb5kdc[9990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:29.818951 10088 catalog_manager.cc:1347] Generated new cluster ID: 4d098de4c5f74658b39125fcf316266a
I20260924 21:12:29.818993 10088 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:12:29.823169 10075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:29.811052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41632 (local address 127.6.210.190:44511)
0924 21:12:29.811226 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:29.811236 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:29.811269 (+ 33us) server_negotiation.cc:407] Connection header received
0924 21:12:29.812527 (+ 1258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:29.812538 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:29.812618 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:29.812769 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:29.813731 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:29.814852 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:29.816017 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:29.816646 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:29.816762 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:29.819483 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:29.819487 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:29.819487 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:29.819498 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:29.821469 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:29.821890 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:29.821893 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:29.821893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:29.822011 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:29.822226 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:29.822227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:29.822227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:29.822522 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:29.822675 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:29.822824 (+ 149us) server_negotiation.cc:299] Negotiation successful
0924 21:12:29.822951 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":94,"threads_started":1}
I20260924 21:12:29.825453 10088 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:12:29.826169 10088 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:12:29.830292 10088 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a5ff97fb05e548e5bb1b2d70a66c4419: Generated new TSK 0
I20260924 21:12:29.830451 10088 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.210.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
I20260924 21:12:29.861558 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.129:0
--local_ip_for_outbound_sockets=127.6.210.129
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:44511
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:34515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:29.944676 10097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:29.944926 10097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:29.944964 10097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:29.946130 10097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:29.946195 10097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:29.946240 10097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.129
I20260924 21:12:29.947721 10097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:34515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:44511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.129
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:29.947897 10097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:29.948076 10097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:29.952881 10102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:29.952869 10103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:29.952868 10105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:29.952885 10097 server_base.cc:1034] running on GCE node
I20260924 21:12:29.953254 10097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:29.953482 10097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:29.954640 10097 hybrid_clock.cc:648] HybridClock initialized: now 1790284349954632 us; error 38 us; skew 500 ppm
Sep 24 21:12:29 dist-test-slave-q276 krb5kdc[9990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284349, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:29.957288 10097 init.cc:377] Logged in from keytab as kudu/127.6.210.129@KRBTEST.COM (short username kudu)
I20260924 21:12:29.957718 10097 webserver.cc:460] Webserver started at http://127.6.210.129:35713/ using document root <none> and password file <none>
I20260924 21:12:29.957939 10097 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:29.958007 10097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:29.958156 10097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:29.959230 10097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/data/instance:
uuid: "322bbe8603b148c299c3cc92387029de"
format_stamp: "Formatted at 2026-09-24 21:12:29 on dist-test-slave-q276"
server_key: "8e273fae587b46bad88207df62a2b882"
server_key_iv: "5895345fd3c3536a5ca2f36b1f2cd8c6"
server_key_version: "encryptionkey@0"
I20260924 21:12:29.959589 10097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance:
uuid: "322bbe8603b148c299c3cc92387029de"
format_stamp: "Formatted at 2026-09-24 21:12:29 on dist-test-slave-q276"
server_key: "8e273fae587b46bad88207df62a2b882"
server_key_iv: "5895345fd3c3536a5ca2f36b1f2cd8c6"
server_key_version: "encryptionkey@0"
I20260924 21:12:29.960877 10097 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 21:12:29.961596 10112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:29.961769 10097 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:12:29.961867 10097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/wal
uuid: "322bbe8603b148c299c3cc92387029de"
format_stamp: "Formatted at 2026-09-24 21:12:29 on dist-test-slave-q276"
server_key: "8e273fae587b46bad88207df62a2b882"
server_key_iv: "5895345fd3c3536a5ca2f36b1f2cd8c6"
server_key_version: "encryptionkey@0"
I20260924 21:12:29.961946 10097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:29.973721 10097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:29.975569 10097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:29.975739 10097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:29.975966 10097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:29.976272 10097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:29.976327 10097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:29.976370 10097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:29.976399 10097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:29.982420 10097 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.129:41647
I20260924 21:12:29.982456 10225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.129:41647 every 8 connection(s)
I20260924 21:12:29.982839 10097 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
I20260924 21:12:29.986586 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 10097
I20260924 21:12:29.986673 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance
I20260924 21:12:29.986959 6986 external_mini_cluster.cc:1428] Setting key a40d158472516c90f2a82df5488892a8
Sep 24 21:12:29 dist-test-slave-q276 krb5kdc[9990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284349, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:29.993314 10075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:29.983582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:50703 (local address 127.6.210.190:44511)
0924 21:12:29.983633 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:29.983634 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:29.983873 (+ 239us) server_negotiation.cc:407] Connection header received
0924 21:12:29.984339 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:29.984340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:29.984351 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:29.984403 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:29.985307 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:29.985902 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:29.987238 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:29.987477 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:29.987564 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:29.990335 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:29.990337 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:29.990337 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:29.990347 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:29.991785 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:29.992356 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:29.992356 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:29.992357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:29.992404 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:29.992696 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:29.992696 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:29.992697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:29.992791 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:29.992838 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:29.993187 (+ 349us) server_negotiation.cc:299] Negotiation successful
0924 21:12:29.993252 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 21:12:29.993598 10228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:29.983595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:44511 (local address 127.6.210.129:50703)
0924 21:12:29.983755 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:29.983789 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:29.984257 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:29.984459 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:29.984463 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:29.985025 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:29.985229 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:29.985234 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:29.985985 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:29.985986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:29.987158 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:29.987160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:29.987572 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:29.987573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:29.987725 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:29.988271 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:29.988283 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:29.990229 (+ 1946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:29.991868 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:29.991874 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:29.991877 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:29.992274 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:29.992473 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:29.992474 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:29.992475 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:29.992627 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:29.992856 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:29.992863 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:29.993045 (+ 182us) client_negotiation.cc:769] Sending connection context
0924 21:12:29.993174 (+ 129us) client_negotiation.cc:240] Negotiation successful
0924 21:12:29.993346 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
I20260924 21:12:29.994166 10226 heartbeater.cc:347] Connected to a master server at 127.6.210.190:44511
I20260924 21:12:29.994292 10226 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:29.994493 10226 heartbeater.cc:511] Master 127.6.210.190:44511 requested a full tablet report, sending...
I20260924 21:12:29.994997 10035 ts_manager.cc:194] Registered new tserver with Master: 322bbe8603b148c299c3cc92387029de (127.6.210.129:41647)
WARNING: no policy specified for kudu/127.6.210.130@KRBTEST.COM; defaulting to no policy
I20260924 21:12:29.995604 10035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.129@KRBTEST.COM'} at 127.6.210.129:50703
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
I20260924 21:12:30.012547 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.130:0
--local_ip_for_outbound_sockets=127.6.210.130
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:44511
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:34515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:30.093428 10231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:30.093652 10231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:30.093677 10231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:30.094903 10231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:30.094945 10231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:30.094972 10231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.130
I20260924 21:12:30.096424 10231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:34515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:44511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.130
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:30.096719 10231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:30.096951 10231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:30.102202 10236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:30.102303 10239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:30.102325 10237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:30.102407 10231 server_base.cc:1034] running on GCE node
I20260924 21:12:30.102685 10231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:30.102968 10231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:30.104136 10231 hybrid_clock.cc:648] HybridClock initialized: now 1790284350104126 us; error 43 us; skew 500 ppm
Sep 24 21:12:30 dist-test-slave-q276 krb5kdc[9990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284350, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:30.106601 10231 init.cc:377] Logged in from keytab as kudu/127.6.210.130@KRBTEST.COM (short username kudu)
I20260924 21:12:30.107017 10231 webserver.cc:460] Webserver started at http://127.6.210.130:46811/ using document root <none> and password file <none>
I20260924 21:12:30.107254 10231 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:30.107321 10231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:30.107468 10231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:30.108510 10231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/data/instance:
uuid: "05fc2c6535f04a53901a3642c6efe3f0"
format_stamp: "Formatted at 2026-09-24 21:12:30 on dist-test-slave-q276"
server_key: "f0873c5253df4f1c4a9729fd3655d57d"
server_key_iv: "f7507b9b701b4562a482564fddab6e07"
server_key_version: "encryptionkey@0"
I20260924 21:12:30.108875 10231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance:
uuid: "05fc2c6535f04a53901a3642c6efe3f0"
format_stamp: "Formatted at 2026-09-24 21:12:30 on dist-test-slave-q276"
server_key: "f0873c5253df4f1c4a9729fd3655d57d"
server_key_iv: "f7507b9b701b4562a482564fddab6e07"
server_key_version: "encryptionkey@0"
I20260924 21:12:30.110167 10231 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:12:30.110906 10246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:30.111112 10231 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:12:30.111204 10231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/wal
uuid: "05fc2c6535f04a53901a3642c6efe3f0"
format_stamp: "Formatted at 2026-09-24 21:12:30 on dist-test-slave-q276"
server_key: "f0873c5253df4f1c4a9729fd3655d57d"
server_key_iv: "f7507b9b701b4562a482564fddab6e07"
server_key_version: "encryptionkey@0"
I20260924 21:12:30.111285 10231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:30.129714 10231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:30.132171 10231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:30.132344 10231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:30.132589 10231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:30.132899 10231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:30.132953 10231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:30.132997 10231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:30.133023 10231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:30.138931 10231 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.130:33147
I20260924 21:12:30.138973 10359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.130:33147 every 8 connection(s)
I20260924 21:12:30.139299 10231 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
Sep 24 21:12:30 dist-test-slave-q276 krb5kdc[9990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284350, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:30.147243 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 10231
I20260924 21:12:30.147372 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance
I20260924 21:12:30.147646 6986 external_mini_cluster.cc:1428] Setting key daad167879f5653660bd03d71c7fff57
I20260924 21:12:30.149089 10075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:30.140043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:56985 (local address 127.6.210.190:44511)
0924 21:12:30.140108 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:30.140110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:30.140324 (+ 214us) server_negotiation.cc:407] Connection header received
0924 21:12:30.140844 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:30.140845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:30.140856 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:30.140975 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:30.141945 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:30.142501 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:30.143533 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:30.143737 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:30.143801 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:30.146039 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:30.146040 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:30.146040 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:30.146047 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:30.147499 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:30.148039 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:30.148040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:30.148040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:30.148102 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:30.148436 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:30.148437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:30.148437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:30.148569 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:30.148639 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:30.148956 (+ 317us) server_negotiation.cc:299] Negotiation successful
0924 21:12:30.149028 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 21:12:30.149430 10362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:30.140025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:44511 (local address 127.6.210.130:56985)
0924 21:12:30.140196 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:30.140237 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:30.140724 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:30.141048 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:30.141052 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:30.141621 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:30.141813 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:30.141817 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:30.142581 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:30.142582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:30.143447 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:30.143450 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:30.143814 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:30.143815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:30.143909 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:30.144438 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:30.144446 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:30.145951 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:30.147574 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:30.147581 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:30.147584 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:30.147947 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:30.148193 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:30.148194 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:30.148195 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:30.148357 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:30.148638 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:30.148644 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:30.148816 (+ 172us) client_negotiation.cc:769] Sending connection context
0924 21:12:30.148938 (+ 122us) client_negotiation.cc:240] Negotiation successful
0924 21:12:30.149154 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":97,"threads_started":1}
I20260924 21:12:30.150214 10360 heartbeater.cc:347] Connected to a master server at 127.6.210.190:44511
I20260924 21:12:30.150343 10360 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:30.150527 10360 heartbeater.cc:511] Master 127.6.210.190:44511 requested a full tablet report, sending...
I20260924 21:12:30.150967 10035 ts_manager.cc:194] Registered new tserver with Master: 05fc2c6535f04a53901a3642c6efe3f0 (127.6.210.130:33147)
I20260924 21:12:30.151505 10035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.130@KRBTEST.COM'} at 127.6.210.130:56985
WARNING: no policy specified for kudu/127.6.210.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
I20260924 21:12:30.173240 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.131:0
--local_ip_for_outbound_sockets=127.6.210.131
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:44511
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:34515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:30.253098 10365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:30.253306 10365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:30.253328 10365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:30.254446 10365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:30.254482 10365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:30.254506 10365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.131
I20260924 21:12:30.255847 10365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:34515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:44511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.131
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:30.256021 10365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:30.256193 10365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:30.261178 10371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:30.261251 10373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:30.261305 10365 server_base.cc:1034] running on GCE node
W20260924 21:12:30.261256 10370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:30.261658 10365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:30.261935 10365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:30.263090 10365 hybrid_clock.cc:648] HybridClock initialized: now 1790284350263070 us; error 33 us; skew 500 ppm
Sep 24 21:12:30 dist-test-slave-q276 krb5kdc[9990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284350, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:30.265699 10365 init.cc:377] Logged in from keytab as kudu/127.6.210.131@KRBTEST.COM (short username kudu)
I20260924 21:12:30.266155 10365 webserver.cc:460] Webserver started at http://127.6.210.131:43121/ using document root <none> and password file <none>
I20260924 21:12:30.266404 10365 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:30.266450 10365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:30.266606 10365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:30.267702 10365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/data/instance:
uuid: "9dfbf500821a48c4a960ae58d30cacc6"
format_stamp: "Formatted at 2026-09-24 21:12:30 on dist-test-slave-q276"
server_key: "8c1899a00b581b2f2c77bb1d06ac2ba0"
server_key_iv: "133cb39a8ecee3d54a3601acf8324344"
server_key_version: "encryptionkey@0"
I20260924 21:12:30.268062 10365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance:
uuid: "9dfbf500821a48c4a960ae58d30cacc6"
format_stamp: "Formatted at 2026-09-24 21:12:30 on dist-test-slave-q276"
server_key: "8c1899a00b581b2f2c77bb1d06ac2ba0"
server_key_iv: "133cb39a8ecee3d54a3601acf8324344"
server_key_version: "encryptionkey@0"
I20260924 21:12:30.269333 10365 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:12:30.270033 10380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:30.270203 10365 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:12:30.270329 10365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/wal
uuid: "9dfbf500821a48c4a960ae58d30cacc6"
format_stamp: "Formatted at 2026-09-24 21:12:30 on dist-test-slave-q276"
server_key: "8c1899a00b581b2f2c77bb1d06ac2ba0"
server_key_iv: "133cb39a8ecee3d54a3601acf8324344"
server_key_version: "encryptionkey@0"
I20260924 21:12:30.270404 10365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:30.286310 10365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:30.288594 10365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:30.288761 10365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:30.288978 10365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:30.289268 10365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:30.289302 10365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:30.289324 10365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:30.289367 10365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:30.295006 10365 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.131:43423
I20260924 21:12:30.295077 10493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.131:43423 every 8 connection(s)
I20260924 21:12:30.295399 10365 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
I20260924 21:12:30.297890 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 10365
I20260924 21:12:30.297989 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance
I20260924 21:12:30.298154 6986 external_mini_cluster.cc:1428] Setting key a632b38a21723105065d91372c86018a
Sep 24 21:12:30 dist-test-slave-q276 krb5kdc[9990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284350, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:30.304416 10075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:30.296080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:33615 (local address 127.6.210.190:44511)
0924 21:12:30.296126 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:30.296128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:30.296302 (+ 174us) server_negotiation.cc:407] Connection header received
0924 21:12:30.296790 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:30.296791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:30.296801 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:30.296852 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:30.297748 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:30.298549 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:30.299673 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:30.299852 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:30.299906 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:30.301869 (+ 1963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:30.301870 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:30.301870 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:30.301877 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:30.303292 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:30.303665 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:30.303665 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:30.303666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:30.303712 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:30.303917 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:30.303918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:30.303918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:30.304012 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:30.304060 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:30.304299 (+ 239us) server_negotiation.cc:299] Negotiation successful
0924 21:12:30.304355 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 21:12:30.304611 10496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:30.296065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:44511 (local address 127.6.210.131:33615)
0924 21:12:30.296201 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:30.296234 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:30.296705 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:30.296905 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:30.296908 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:30.297475 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:30.297666 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:30.297672 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:30.298628 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:30.298629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:30.299593 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:30.299595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:30.299922 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:30.299923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:30.300018 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:30.300465 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:30.300472 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:30.301775 (+ 1303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:30.303368 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:30.303374 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:30.303376 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:30.303599 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:30.303769 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:30.303770 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:30.303770 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:30.303864 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:30.304063 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:30.304067 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:30.304193 (+ 126us) client_negotiation.cc:769] Sending connection context
0924 21:12:30.304275 (+ 82us) client_negotiation.cc:240] Negotiation successful
0924 21:12:30.304396 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":70,"threads_started":1}
I20260924 21:12:30.305063 10494 heartbeater.cc:347] Connected to a master server at 127.6.210.190:44511
I20260924 21:12:30.305184 10494 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:30.305361 10494 heartbeater.cc:511] Master 127.6.210.190:44511 requested a full tablet report, sending...
I20260924 21:12:30.305717 10035 ts_manager.cc:194] Registered new tserver with Master: 9dfbf500821a48c4a960ae58d30cacc6 (127.6.210.131:43423)
I20260924 21:12:30.306090 10035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.131@KRBTEST.COM'} at 127.6.210.131:33615
I20260924 21:12:30.309882 6986 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 24 21:12:30 dist-test-slave-q276 krb5kdc[9990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284350, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 24 21:12:30 dist-test-slave-q276 krb5kdc[9990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284350, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:30.331841 10075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:30.324888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41642 (local address 127.6.210.190:44511)
0924 21:12:30.324955 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:30.324958 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:30.325065 (+ 107us) server_negotiation.cc:407] Connection header received
0924 21:12:30.325178 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:30.325179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:30.325196 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:30.325260 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:30.325795 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:30.326327 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:30.327044 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:30.327238 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:30.327311 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:30.329403 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:30.329404 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:30.329404 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:30.329412 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:30.330701 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:30.331021 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:30.331022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:30.331023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:30.331080 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:30.331296 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:30.331297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:30.331297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:30.331438 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:30.331544 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:30.331664 (+ 120us) server_negotiation.cc:299] Negotiation successful
0924 21:12:30.331740 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260924 21:12:30.332067 10035 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:41642
I20260924 21:12:30.332525 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 10097
I20260924 21:12:30.337090 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 10231
I20260924 21:12:30.341391 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 10365
I20260924 21:12:30.345515 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 10003
2026-09-24T21:12:30Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1827 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:12:30 dist-test-slave-q276 krb5kdc[10506](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:12:30 dist-test-slave-q276 krb5kdc[10506](info): set up 2 sockets
Sep 24 21:12:30 dist-test-slave-q276 krb5kdc[10506](info): commencing operation
krb5kdc: starting...
W20260924 21:12:31.391307 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:12:31 dist-test-slave-q276 krb5kdc[10506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:12:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:12:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.210.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
I20260924 21:12:31.514650 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:41725
--webserver_interface=127.6.210.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:45803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:41725
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:31.595371 10519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:31.595698 10519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:31.595758 10519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:31.597376 10519 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:12:31.597465 10519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:31.597512 10519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:31.597554 10519 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:12:31.597596 10519 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:12:31.600140 10519 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:45803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:41725
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:41725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:31.600419 10519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:31.600696 10519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:31.606230 10527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:31.606357 10524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:31.606209 10525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:31.606602 10519 server_base.cc:1034] running on GCE node
I20260924 21:12:31.606863 10519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:31.607187 10519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:31.608357 10519 hybrid_clock.cc:648] HybridClock initialized: now 1790284351608334 us; error 41 us; skew 500 ppm
Sep 24 21:12:31 dist-test-slave-q276 krb5kdc[10506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284351, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:31.610908 10519 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:12:31.611351 10519 webserver.cc:460] Webserver started at http://127.6.210.190:38553/ using document root <none> and password file <none>
I20260924 21:12:31.611599 10519 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:31.611652 10519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:31.611773 10519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:31.612905 10519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "c70f3bcf655a4e4d8dd3996ca9376ab6"
format_stamp: "Formatted at 2026-09-24 21:12:31 on dist-test-slave-q276"
server_key: "af7e29f8efd90959280dea9b46488f45"
server_key_iv: "71e8021c0fe6e259ba027216a37b639a"
server_key_version: "encryptionkey@0"
I20260924 21:12:31.613300 10519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "c70f3bcf655a4e4d8dd3996ca9376ab6"
format_stamp: "Formatted at 2026-09-24 21:12:31 on dist-test-slave-q276"
server_key: "af7e29f8efd90959280dea9b46488f45"
server_key_iv: "71e8021c0fe6e259ba027216a37b639a"
server_key_version: "encryptionkey@0"
I20260924 21:12:31.614703 10519 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:12:31.615479 10534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:31.615679 10519 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 21:12:31.615792 10519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "c70f3bcf655a4e4d8dd3996ca9376ab6"
format_stamp: "Formatted at 2026-09-24 21:12:31 on dist-test-slave-q276"
server_key: "af7e29f8efd90959280dea9b46488f45"
server_key_iv: "71e8021c0fe6e259ba027216a37b639a"
server_key_version: "encryptionkey@0"
I20260924 21:12:31.615869 10519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:31.658337 10519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:31.660514 10519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:31.660701 10519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:31.664739 10519 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:41725
I20260924 21:12:31.664988 10586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:41725 every 8 connection(s)
I20260924 21:12:31.665180 10519 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:12:31.665905 10587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:31.668157 10587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c70f3bcf655a4e4d8dd3996ca9376ab6: Bootstrap starting.
I20260924 21:12:31.668699 10587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c70f3bcf655a4e4d8dd3996ca9376ab6: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:31.668933 10587 log.cc:824] T 00000000000000000000000000000000 P c70f3bcf655a4e4d8dd3996ca9376ab6: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:31.669605 10587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c70f3bcf655a4e4d8dd3996ca9376ab6: No bootstrap required, opened a new log
I20260924 21:12:31.670516 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 10519
I20260924 21:12:31.670632 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:12:31.670830 6986 external_mini_cluster.cc:1428] Setting key 855403d2c5f323730227c0b16c62a56f
I20260924 21:12:31.672324 10587 raft_consensus.cc:348] T 00000000000000000000000000000000 P c70f3bcf655a4e4d8dd3996ca9376ab6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70f3bcf655a4e4d8dd3996ca9376ab6" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 41725 } }
I20260924 21:12:31.672502 10587 raft_consensus.cc:374] T 00000000000000000000000000000000 P c70f3bcf655a4e4d8dd3996ca9376ab6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:31.672538 10587 raft_consensus.cc:729] T 00000000000000000000000000000000 P c70f3bcf655a4e4d8dd3996ca9376ab6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c70f3bcf655a4e4d8dd3996ca9376ab6, State: Initialized, Role: FOLLOWER
I20260924 21:12:31.672732 10587 consensus_queue.cc:260] T 00000000000000000000000000000000 P c70f3bcf655a4e4d8dd3996ca9376ab6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70f3bcf655a4e4d8dd3996ca9376ab6" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 41725 } }
I20260924 21:12:31.672813 10587 raft_consensus.cc:388] T 00000000000000000000000000000000 P c70f3bcf655a4e4d8dd3996ca9376ab6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:12:31.672837 10587 raft_consensus.cc:482] T 00000000000000000000000000000000 P c70f3bcf655a4e4d8dd3996ca9376ab6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:12:31.672904 10587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c70f3bcf655a4e4d8dd3996ca9376ab6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:31.673629 10587 raft_consensus.cc:504] T 00000000000000000000000000000000 P c70f3bcf655a4e4d8dd3996ca9376ab6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70f3bcf655a4e4d8dd3996ca9376ab6" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 41725 } }
I20260924 21:12:31.673734 10587 leader_election.cc:302] T 00000000000000000000000000000000 P c70f3bcf655a4e4d8dd3996ca9376ab6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c70f3bcf655a4e4d8dd3996ca9376ab6; no voters:
I20260924 21:12:31.673882 10587 leader_election.cc:288] T 00000000000000000000000000000000 P c70f3bcf655a4e4d8dd3996ca9376ab6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:12:31.673981 10592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c70f3bcf655a4e4d8dd3996ca9376ab6 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:31.674165 10592 raft_consensus.cc:686] T 00000000000000000000000000000000 P c70f3bcf655a4e4d8dd3996ca9376ab6 [term 1 LEADER]: Becoming Leader. State: Replica: c70f3bcf655a4e4d8dd3996ca9376ab6, State: Running, Role: LEADER
I20260924 21:12:31.674315 10592 consensus_queue.cc:237] T 00000000000000000000000000000000 P c70f3bcf655a4e4d8dd3996ca9376ab6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70f3bcf655a4e4d8dd3996ca9376ab6" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 41725 } }
I20260924 21:12:31.674155 10587 sys_catalog.cc:564] T 00000000000000000000000000000000 P c70f3bcf655a4e4d8dd3996ca9376ab6 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:12:31.674685 10593 sys_catalog.cc:455] T 00000000000000000000000000000000 P c70f3bcf655a4e4d8dd3996ca9376ab6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c70f3bcf655a4e4d8dd3996ca9376ab6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70f3bcf655a4e4d8dd3996ca9376ab6" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 41725 } } }
I20260924 21:12:31.674952 10593 sys_catalog.cc:458] T 00000000000000000000000000000000 P c70f3bcf655a4e4d8dd3996ca9376ab6 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:31.674881 10594 sys_catalog.cc:455] T 00000000000000000000000000000000 P c70f3bcf655a4e4d8dd3996ca9376ab6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c70f3bcf655a4e4d8dd3996ca9376ab6. Latest consensus state: current_term: 1 leader_uuid: "c70f3bcf655a4e4d8dd3996ca9376ab6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70f3bcf655a4e4d8dd3996ca9376ab6" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 41725 } } }
I20260924 21:12:31.675243 10594 sys_catalog.cc:458] T 00000000000000000000000000000000 P c70f3bcf655a4e4d8dd3996ca9376ab6 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:31.675395 10597 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:12:31.675917 10597 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:12:31.677896 10597 catalog_manager.cc:1347] Generated new cluster ID: 9534366cd00342c4b0b6728c7496e80b
I20260924 21:12:31.677955 10597 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 21:12:31 dist-test-slave-q276 krb5kdc[10506](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:31.683339 10591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:31.671395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37948 (local address 127.6.210.190:41725)
0924 21:12:31.671617 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:31.671628 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:31.671665 (+ 37us) server_negotiation.cc:407] Connection header received
0924 21:12:31.672284 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:31.672293 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:31.672360 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:31.672522 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:31.673290 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:31.674699 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:31.675892 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:31.676662 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:31.676766 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:31.679518 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:31.679522 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:31.679522 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:31.679534 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:31.681492 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:31.681997 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:31.682000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:31.682000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:31.682140 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:31.682398 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:31.682399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:31.682399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:31.682705 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:31.682866 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:31.682989 (+ 123us) server_negotiation.cc:299] Negotiation successful
0924 21:12:31.683134 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":100,"threads_started":1}
I20260924 21:12:31.710929 10597 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:12:31.711490 10597 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:12:31.718926 10597 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c70f3bcf655a4e4d8dd3996ca9376ab6: Generated new TSK 0
I20260924 21:12:31.719113 10597 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.210.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
I20260924 21:12:31.751130 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.129:0
--local_ip_for_outbound_sockets=127.6.210.129
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:41725
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:45803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:31.831506 10613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:31.831710 10613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:31.831734 10613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:31.832826 10613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:31.832860 10613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:31.832885 10613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.129
I20260924 21:12:31.834385 10613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:45803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:41725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.129
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:31.834573 10613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:31.834750 10613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:31.839684 10619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:31.839777 10618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:31.839821 10613 server_base.cc:1034] running on GCE node
W20260924 21:12:31.839787 10621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:31.840112 10613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:31.840390 10613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:31.841557 10613 hybrid_clock.cc:648] HybridClock initialized: now 1790284351841541 us; error 47 us; skew 500 ppm
Sep 24 21:12:31 dist-test-slave-q276 krb5kdc[10506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284351, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:31.844158 10613 init.cc:377] Logged in from keytab as kudu/127.6.210.129@KRBTEST.COM (short username kudu)
I20260924 21:12:31.844558 10613 webserver.cc:460] Webserver started at http://127.6.210.129:37097/ using document root <none> and password file <none>
I20260924 21:12:31.844744 10613 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:31.844784 10613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:31.844872 10613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:31.845882 10613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-0/data/instance:
uuid: "186e4203c97d4a7589bc3b41a7694266"
format_stamp: "Formatted at 2026-09-24 21:12:31 on dist-test-slave-q276"
server_key: "105e6abaec942429f2c3e36641a2c59f"
server_key_iv: "4776c9f0410b5da035eaf9a6cd474b3c"
server_key_version: "encryptionkey@0"
I20260924 21:12:31.846196 10613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance:
uuid: "186e4203c97d4a7589bc3b41a7694266"
format_stamp: "Formatted at 2026-09-24 21:12:31 on dist-test-slave-q276"
server_key: "105e6abaec942429f2c3e36641a2c59f"
server_key_iv: "4776c9f0410b5da035eaf9a6cd474b3c"
server_key_version: "encryptionkey@0"
I20260924 21:12:31.847491 10613 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:12:31.848240 10628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:31.848388 10613 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:12:31.848475 10613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-0/wal
uuid: "186e4203c97d4a7589bc3b41a7694266"
format_stamp: "Formatted at 2026-09-24 21:12:31 on dist-test-slave-q276"
server_key: "105e6abaec942429f2c3e36641a2c59f"
server_key_iv: "4776c9f0410b5da035eaf9a6cd474b3c"
server_key_version: "encryptionkey@0"
I20260924 21:12:31.848531 10613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:31.857120 10613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:31.858945 10613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:31.859053 10613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:31.859216 10613 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:31.859454 10613 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:31.859490 10613 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:31.859511 10613 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:31.859524 10613 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:31.864985 10613 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.129:42195
I20260924 21:12:31.865104 10741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.129:42195 every 8 connection(s)
I20260924 21:12:31.865381 10613 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
I20260924 21:12:31.865525 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 10613
I20260924 21:12:31.865633 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance
I20260924 21:12:31.865847 6986 external_mini_cluster.cc:1428] Setting key 3a744090c6be0e03d8e9c94c6b88efb5
Sep 24 21:12:31 dist-test-slave-q276 krb5kdc[10506](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284351, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.210.130@KRBTEST.COM; defaulting to no policy
I20260924 21:12:31.876142 10591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:31.866110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:40163 (local address 127.6.210.190:41725)
0924 21:12:31.866158 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:31.866160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:31.866404 (+ 244us) server_negotiation.cc:407] Connection header received
0924 21:12:31.867393 (+ 989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:31.867395 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:31.867411 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:31.867478 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:31.868346 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:31.868865 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:31.870445 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:31.870651 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:31.870706 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:31.873110 (+ 2404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:31.873112 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:31.873112 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:31.873119 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:31.874584 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:31.875060 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:31.875062 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:31.875062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:31.875125 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:31.875420 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:31.875421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:31.875422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:31.875535 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:31.875613 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:31.875877 (+ 264us) server_negotiation.cc:299] Negotiation successful
0924 21:12:31.876043 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 21:12:31.876299 10744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:31.866107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:41725 (local address 127.6.210.129:40163)
0924 21:12:31.866256 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:31.866319 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:31.867280 (+ 961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:31.867537 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:31.867540 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:31.867941 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:31.868244 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:31.868250 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:31.868956 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:31.868958 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:31.870360 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:31.870363 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:31.870743 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:31.870745 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:31.870915 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:31.871451 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:31.871462 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:31.872996 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:31.874677 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:31.874683 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:31.874686 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:31.874970 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:31.875216 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:31.875217 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:31.875217 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:31.875344 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:31.875606 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:31.875612 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:31.875749 (+ 137us) client_negotiation.cc:769] Sending connection context
0924 21:12:31.875856 (+ 107us) client_negotiation.cc:240] Negotiation successful
0924 21:12:31.876045 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.130@KRBTEST.COM" created.
I20260924 21:12:31.876807 10742 heartbeater.cc:347] Connected to a master server at 127.6.210.190:41725
I20260924 21:12:31.876900 10742 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:31.877104 10742 heartbeater.cc:511] Master 127.6.210.190:41725 requested a full tablet report, sending...
I20260924 21:12:31.877665 10551 ts_manager.cc:194] Registered new tserver with Master: 186e4203c97d4a7589bc3b41a7694266 (127.6.210.129:42195)
I20260924 21:12:31.878504 10551 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.129@KRBTEST.COM'} at 127.6.210.129:40163
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
I20260924 21:12:31.891196 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.130:0
--local_ip_for_outbound_sockets=127.6.210.130
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:41725
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:45803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:31.975332 10747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:31.975569 10747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:31.975607 10747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:31.976912 10747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:31.976979 10747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:31.977023 10747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.130
I20260924 21:12:31.978539 10747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:45803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:41725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.130
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:31.978803 10747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:31.979033 10747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:31.984901 10753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:31.984910 10752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:31.985073 10747 server_base.cc:1034] running on GCE node
W20260924 21:12:31.985133 10755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:31.985617 10747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:31.985917 10747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:31.987102 10747 hybrid_clock.cc:648] HybridClock initialized: now 1790284351987084 us; error 38 us; skew 500 ppm
Sep 24 21:12:31 dist-test-slave-q276 krb5kdc[10506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284351, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:31.989802 10747 init.cc:377] Logged in from keytab as kudu/127.6.210.130@KRBTEST.COM (short username kudu)
I20260924 21:12:31.990242 10747 webserver.cc:460] Webserver started at http://127.6.210.130:44343/ using document root <none> and password file <none>
I20260924 21:12:31.990541 10747 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:31.990612 10747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:31.990757 10747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:31.991823 10747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-1/data/instance:
uuid: "1fdfb1e113a04df0b9d5ff923804bb5b"
format_stamp: "Formatted at 2026-09-24 21:12:31 on dist-test-slave-q276"
server_key: "f85d966fb91e5010e8f6354cb97eff33"
server_key_iv: "5a7512eed23088f31bc4f01c59739e15"
server_key_version: "encryptionkey@0"
I20260924 21:12:31.992202 10747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance:
uuid: "1fdfb1e113a04df0b9d5ff923804bb5b"
format_stamp: "Formatted at 2026-09-24 21:12:31 on dist-test-slave-q276"
server_key: "f85d966fb91e5010e8f6354cb97eff33"
server_key_iv: "5a7512eed23088f31bc4f01c59739e15"
server_key_version: "encryptionkey@0"
I20260924 21:12:31.993515 10747 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:12:31.994181 10762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:31.994370 10747 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:12:31.994480 10747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-1/wal
uuid: "1fdfb1e113a04df0b9d5ff923804bb5b"
format_stamp: "Formatted at 2026-09-24 21:12:31 on dist-test-slave-q276"
server_key: "f85d966fb91e5010e8f6354cb97eff33"
server_key_iv: "5a7512eed23088f31bc4f01c59739e15"
server_key_version: "encryptionkey@0"
I20260924 21:12:31.994552 10747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:32.005193 10747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:32.007252 10747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:32.007416 10747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:32.007652 10747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:32.007965 10747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:32.008019 10747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:32.008059 10747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:32.008086 10747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:32.013782 10747 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.130:44763
I20260924 21:12:32.013878 10875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.130:44763 every 8 connection(s)
I20260924 21:12:32.014174 10747 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
I20260924 21:12:32.015781 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 10747
I20260924 21:12:32.015895 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance
I20260924 21:12:32.016065 6986 external_mini_cluster.cc:1428] Setting key d277bc4593347a3ac2dc1f669354d519
Sep 24 21:12:32 dist-test-slave-q276 krb5kdc[10506](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284351, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.210.131@KRBTEST.COM; defaulting to no policy
I20260924 21:12:32.025507 10591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:32.014929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:41467 (local address 127.6.210.190:41725)
0924 21:12:32.015059 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:32.015061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:32.015175 (+ 114us) server_negotiation.cc:407] Connection header received
0924 21:12:32.016155 (+ 980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:32.016156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:32.016174 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:32.016249 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:32.017025 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:32.017782 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:32.019333 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:32.019534 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:32.019663 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:32.022331 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:32.022333 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:32.022333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:32.022340 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:32.023851 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:32.024378 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:32.024379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:32.024379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:32.024450 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:32.024748 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:32.024749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:32.024750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:32.024876 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:32.024976 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:32.025261 (+ 285us) server_negotiation.cc:299] Negotiation successful
0924 21:12:32.025358 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260924 21:12:32.025655 10878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:32.014881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:41725 (local address 127.6.210.130:41467)
0924 21:12:32.015035 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:32.015076 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:32.016035 (+ 959us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:32.016388 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:32.016391 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:32.016684 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:32.016930 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:32.016935 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:32.017873 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:32.017874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:32.019268 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:32.019271 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:32.019676 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:32.019677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:32.019844 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:32.020455 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:32.020463 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:32.022160 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:32.024023 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:32.024031 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:32.024034 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:32.024280 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:32.024534 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:32.024534 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:32.024535 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:32.024673 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:32.024969 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:32.024976 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:32.025123 (+ 147us) client_negotiation.cc:769] Sending connection context
0924 21:12:32.025241 (+ 118us) client_negotiation.cc:240] Negotiation successful
0924 21:12:32.025416 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":88,"threads_started":1}
I20260924 21:12:32.026140 10876 heartbeater.cc:347] Connected to a master server at 127.6.210.190:41725
I20260924 21:12:32.026229 10876 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:32.026469 10876 heartbeater.cc:511] Master 127.6.210.190:41725 requested a full tablet report, sending...
I20260924 21:12:32.026957 10551 ts_manager.cc:194] Registered new tserver with Master: 1fdfb1e113a04df0b9d5ff923804bb5b (127.6.210.130:44763)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.131@KRBTEST.COM" created.
I20260924 21:12:32.027477 10551 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.130@KRBTEST.COM'} at 127.6.210.130:41467
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
I20260924 21:12:32.041510 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.131:0
--local_ip_for_outbound_sockets=127.6.210.131
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:41725
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:45803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:32.121672 10881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:32.121869 10881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:32.121892 10881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:32.122983 10881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:32.123018 10881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:32.123044 10881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.131
I20260924 21:12:32.124434 10881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:45803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:41725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.131
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:32.124610 10881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:32.124778 10881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 21:12:32.129719 10881 server_base.cc:1034] running on GCE node
W20260924 21:12:32.129678 10887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:32.129699 10889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:32.129688 10886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:32.130152 10881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:32.130549 10881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:32.131727 10881 hybrid_clock.cc:648] HybridClock initialized: now 1790284352131714 us; error 26 us; skew 500 ppm
Sep 24 21:12:32 dist-test-slave-q276 krb5kdc[10506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284352, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:32.134567 10881 init.cc:377] Logged in from keytab as kudu/127.6.210.131@KRBTEST.COM (short username kudu)
I20260924 21:12:32.134980 10881 webserver.cc:460] Webserver started at http://127.6.210.131:45857/ using document root <none> and password file <none>
I20260924 21:12:32.135203 10881 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:32.135272 10881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:32.135427 10881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:32.136444 10881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-2/data/instance:
uuid: "a69d73c69d1f47e48c02d24a7ff908cb"
format_stamp: "Formatted at 2026-09-24 21:12:32 on dist-test-slave-q276"
server_key: "8faeaa1fbd420757762d4d2b8845a6c3"
server_key_iv: "2583b09762cb9bdcb814dc616ba9203a"
server_key_version: "encryptionkey@0"
I20260924 21:12:32.136830 10881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance:
uuid: "a69d73c69d1f47e48c02d24a7ff908cb"
format_stamp: "Formatted at 2026-09-24 21:12:32 on dist-test-slave-q276"
server_key: "8faeaa1fbd420757762d4d2b8845a6c3"
server_key_iv: "2583b09762cb9bdcb814dc616ba9203a"
server_key_version: "encryptionkey@0"
I20260924 21:12:32.138114 10881 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:12:32.138865 10896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:32.139034 10881 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:32.139133 10881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-2/wal
uuid: "a69d73c69d1f47e48c02d24a7ff908cb"
format_stamp: "Formatted at 2026-09-24 21:12:32 on dist-test-slave-q276"
server_key: "8faeaa1fbd420757762d4d2b8845a6c3"
server_key_iv: "2583b09762cb9bdcb814dc616ba9203a"
server_key_version: "encryptionkey@0"
I20260924 21:12:32.139220 10881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:32.151886 10881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:32.153707 10881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:32.153864 10881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:32.154088 10881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:32.154448 10881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:32.154500 10881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:32.154541 10881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:32.154577 10881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:32.160126 10881 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.131:43095
I20260924 21:12:32.160228 11009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.131:43095 every 8 connection(s)
I20260924 21:12:32.160522 10881 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
I20260924 21:12:32.166234 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 10881
I20260924 21:12:32.166410 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance
Sep 24 21:12:32 dist-test-slave-q276 krb5kdc[10506](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284352, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:32.166601 6986 external_mini_cluster.cc:1428] Setting key a584803597682d7d5c076701a26f8ce9
I20260924 21:12:32.169456 10591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:32.161263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:48557 (local address 127.6.210.190:41725)
0924 21:12:32.161307 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:32.161308 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:32.161497 (+ 189us) server_negotiation.cc:407] Connection header received
0924 21:12:32.162481 (+ 984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:32.162483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:32.162492 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:32.162546 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:32.163120 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:32.163591 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:32.164537 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:32.164718 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:32.164770 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:32.167098 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:32.167099 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:32.167099 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:32.167107 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:32.168402 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:32.168761 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:32.168762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:32.168762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:32.168809 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:32.168964 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:32.168965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:32.168965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:32.169052 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:32.169100 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:32.169336 (+ 236us) server_negotiation.cc:299] Negotiation successful
0924 21:12:32.169393 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 21:12:32.169718 11012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:32.161227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:41725 (local address 127.6.210.131:48557)
0924 21:12:32.161372 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:32.161406 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:32.162393 (+ 987us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:32.162603 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:32.162606 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:32.162854 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:32.163037 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:32.163041 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:32.163666 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:32.163666 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:32.164463 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:32.164464 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:32.164789 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:32.164790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:32.164883 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:32.165425 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:32.165434 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:32.167025 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:32.168476 (+ 1451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:32.168481 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:32.168483 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:32.168695 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:32.168849 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:32.168850 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:32.168850 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:32.168930 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:32.169107 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:32.169111 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:32.169227 (+ 116us) client_negotiation.cc:769] Sending connection context
0924 21:12:32.169307 (+ 80us) client_negotiation.cc:240] Negotiation successful
0924 21:12:32.169474 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":88,"threads_started":1}
I20260924 21:12:32.170250 11010 heartbeater.cc:347] Connected to a master server at 127.6.210.190:41725
I20260924 21:12:32.170379 11010 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:32.170545 11010 heartbeater.cc:511] Master 127.6.210.190:41725 requested a full tablet report, sending...
I20260924 21:12:32.170873 10551 ts_manager.cc:194] Registered new tserver with Master: a69d73c69d1f47e48c02d24a7ff908cb (127.6.210.131:43095)
I20260924 21:12:32.171240 10551 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.131@KRBTEST.COM'} at 127.6.210.131:48557
I20260924 21:12:32.178251 6986 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 24 21:12:32 dist-test-slave-q276 krb5kdc[10506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 21:12:32 dist-test-slave-q276 krb5kdc[10506](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.129@KRBTEST.COM
I20260924 21:12:32.200393 11017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:32.193082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37558 (local address 127.6.210.129:42195)
0924 21:12:32.193249 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:32.193254 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:32.193264 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:12:32.193290 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:32.193292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:32.193306 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:32.193370 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:32.193866 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:32.194640 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:32.195306 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:32.195648 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:32.195705 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:32.197961 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:32.197968 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:32.197969 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:32.197975 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:32.199442 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:32.199759 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:32.199761 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:32.199762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:32.199819 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:32.199977 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:32.199978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:32.199978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:32.200098 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:32.200197 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:32.200259 (+ 62us) server_negotiation.cc:299] Negotiation successful
0924 21:12:32.200317 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":77,"threads_started":1}
Sep 24 21:12:32 dist-test-slave-q276 krb5kdc[10506](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:32.208076 10591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:32.201682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37954 (local address 127.6.210.190:41725)
0924 21:12:32.201745 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:32.201747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:32.201785 (+ 38us) server_negotiation.cc:407] Connection header received
0924 21:12:32.201848 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:32.201849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:32.201856 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:32.201895 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:32.202420 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:32.202899 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:32.203528 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:32.203707 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:32.203754 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:32.205887 (+ 2133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:32.205889 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:32.205889 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:32.205896 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:32.207174 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:32.207470 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:32.207471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:32.207471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:32.207516 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:32.207673 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:32.207673 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:32.207673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:32.207804 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:32.207876 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:32.207956 (+ 80us) server_negotiation.cc:299] Negotiation successful
0924 21:12:32.208013 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260924 21:12:32.208344 10551 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37954
I20260924 21:12:32.208720 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 10613
I20260924 21:12:32.214600 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 10747
I20260924 21:12:32.218858 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 10881
I20260924 21:12:32.223163 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 10519
2026-09-24T21:12:32Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1877 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:12:32 dist-test-slave-q276 krb5kdc[11022](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:12:32 dist-test-slave-q276 krb5kdc[11022](info): set up 2 sockets
Sep 24 21:12:32 dist-test-slave-q276 krb5kdc[11022](info): commencing operation
krb5kdc: starting...
W20260924 21:12:33.295459 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:12:33 dist-test-slave-q276 krb5kdc[11022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:12:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:12:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.210.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
I20260924 21:12:33.413275 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:37239
--webserver_interface=127.6.210.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:45687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:37239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:33.497099 11035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:33.497361 11035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:33.497398 11035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:33.498570 11035 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:12:33.498647 11035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:33.498670 11035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:33.498692 11035 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:12:33.498721 11035 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:12:33.500207 11035 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:45687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:37239
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:37239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:33.500483 11035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:33.500708 11035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:33.505681 11043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:33.505681 11040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:33.505812 11041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:33.506047 11035 server_base.cc:1034] running on GCE node
I20260924 21:12:33.506400 11035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:33.506721 11035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:33.507866 11035 hybrid_clock.cc:648] HybridClock initialized: now 1790284353507845 us; error 36 us; skew 500 ppm
Sep 24 21:12:33 dist-test-slave-q276 krb5kdc[11022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284353, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:33.515691 11035 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:12:33.516060 11035 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:33.516111 11035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:33.516211 11035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:33.517257 11035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "6c507efc77aa443b92001d330bfa5953"
format_stamp: "Formatted at 2026-09-24 21:12:33 on dist-test-slave-q276"
server_key: "a4c11fd898ab453d9fbf2e884a04f3b1"
server_key_iv: "d472b765f842a88b87e2ebad89012e18"
server_key_version: "encryptionkey@0"
I20260924 21:12:33.517556 11035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "6c507efc77aa443b92001d330bfa5953"
format_stamp: "Formatted at 2026-09-24 21:12:33 on dist-test-slave-q276"
server_key: "a4c11fd898ab453d9fbf2e884a04f3b1"
server_key_iv: "d472b765f842a88b87e2ebad89012e18"
server_key_version: "encryptionkey@0"
I20260924 21:12:33.518877 11035 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:12:33.519533 11049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:33.519719 11035 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:12:33.519827 11035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "6c507efc77aa443b92001d330bfa5953"
format_stamp: "Formatted at 2026-09-24 21:12:33 on dist-test-slave-q276"
server_key: "a4c11fd898ab453d9fbf2e884a04f3b1"
server_key_iv: "d472b765f842a88b87e2ebad89012e18"
server_key_version: "encryptionkey@0"
I20260924 21:12:33.519901 11035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:33.531236 11035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:33.533536 11035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:33.533703 11035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:33.537761 11035 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:37239
I20260924 21:12:33.537948 11101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:37239 every 8 connection(s)
I20260924 21:12:33.538139 11035 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:12:33.539033 11102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:33.541271 11102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c507efc77aa443b92001d330bfa5953: Bootstrap starting.
I20260924 21:12:33.541859 11102 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c507efc77aa443b92001d330bfa5953: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:33.542162 11102 log.cc:824] T 00000000000000000000000000000000 P 6c507efc77aa443b92001d330bfa5953: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:33.542850 11102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c507efc77aa443b92001d330bfa5953: No bootstrap required, opened a new log
I20260924 21:12:33.544435 11102 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c507efc77aa443b92001d330bfa5953 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c507efc77aa443b92001d330bfa5953" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37239 } }
I20260924 21:12:33.544569 11102 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c507efc77aa443b92001d330bfa5953 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:33.544592 11102 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c507efc77aa443b92001d330bfa5953 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c507efc77aa443b92001d330bfa5953, State: Initialized, Role: FOLLOWER
I20260924 21:12:33.544755 11102 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c507efc77aa443b92001d330bfa5953 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c507efc77aa443b92001d330bfa5953" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37239 } }
I20260924 21:12:33.544849 11102 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c507efc77aa443b92001d330bfa5953 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:12:33.544890 11102 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c507efc77aa443b92001d330bfa5953 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:12:33.544945 11102 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c507efc77aa443b92001d330bfa5953 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:33.545717 11102 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c507efc77aa443b92001d330bfa5953 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c507efc77aa443b92001d330bfa5953" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37239 } }
I20260924 21:12:33.545835 11102 leader_election.cc:302] T 00000000000000000000000000000000 P 6c507efc77aa443b92001d330bfa5953 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c507efc77aa443b92001d330bfa5953; no voters:
I20260924 21:12:33.546008 11102 leader_election.cc:288] T 00000000000000000000000000000000 P 6c507efc77aa443b92001d330bfa5953 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:12:33.546068 11105 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c507efc77aa443b92001d330bfa5953 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:33.546303 11102 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c507efc77aa443b92001d330bfa5953 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:12:33.546355 11105 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c507efc77aa443b92001d330bfa5953 [term 1 LEADER]: Becoming Leader. State: Replica: 6c507efc77aa443b92001d330bfa5953, State: Running, Role: LEADER
I20260924 21:12:33.546451 11105 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c507efc77aa443b92001d330bfa5953 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c507efc77aa443b92001d330bfa5953" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37239 } }
I20260924 21:12:33.546797 11105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c507efc77aa443b92001d330bfa5953 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c507efc77aa443b92001d330bfa5953. Latest consensus state: current_term: 1 leader_uuid: "6c507efc77aa443b92001d330bfa5953" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c507efc77aa443b92001d330bfa5953" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37239 } } }
I20260924 21:12:33.546872 11105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c507efc77aa443b92001d330bfa5953 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:33.546897 11106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c507efc77aa443b92001d330bfa5953 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c507efc77aa443b92001d330bfa5953" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c507efc77aa443b92001d330bfa5953" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37239 } } }
I20260924 21:12:33.546993 11106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c507efc77aa443b92001d330bfa5953 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:33.547075 11111 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:12:33.547609 11111 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:12:33.548174 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 11035
I20260924 21:12:33.548297 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:12:33.548463 6986 external_mini_cluster.cc:1428] Setting key 8eeb35f2b2816f17b59504a2602ed99b
I20260924 21:12:33.551250 11111 catalog_manager.cc:1347] Generated new cluster ID: 70b1df5863824f4ab142488e3a41a4b8
I20260924 21:12:33.551302 11111 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 21:12:33 dist-test-slave-q276 krb5kdc[11022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:33.559669 11124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:33.548931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42234 (local address 127.6.210.190:37239)
0924 21:12:33.549107 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:33.549117 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:33.549153 (+ 36us) server_negotiation.cc:407] Connection header received
0924 21:12:33.549266 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:33.549274 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:33.549328 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:33.549471 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:33.550024 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:33.551444 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:33.552492 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:33.553130 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:33.553244 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:33.556045 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:33.556049 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:33.556049 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:33.556067 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:33.557988 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:33.558418 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:33.558421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:33.558421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:33.558535 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:33.558757 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:33.558757 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:33.558758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:33.559046 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:33.559195 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:33.559315 (+ 120us) server_negotiation.cc:299] Negotiation successful
0924 21:12:33.559441 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":87,"threads_started":1}
I20260924 21:12:33.584731 11111 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:12:33.585283 11111 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:12:33.590569 11111 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c507efc77aa443b92001d330bfa5953: Generated new TSK 0
I20260924 21:12:33.590775 11111 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.210.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
I20260924 21:12:33.624650 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.129:0
--local_ip_for_outbound_sockets=127.6.210.129
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:37239
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:45687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:33.711805 11128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:33.712001 11128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:33.712025 11128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:33.713114 11128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:33.713147 11128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:33.713173 11128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.129
I20260924 21:12:33.714612 11128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:45687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:37239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.129
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:33.714797 11128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:33.714962 11128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:33.719842 11136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:33.719938 11134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:33.719867 11133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:33.720132 11128 server_base.cc:1034] running on GCE node
I20260924 21:12:33.720327 11128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:33.720510 11128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:33.721642 11128 hybrid_clock.cc:648] HybridClock initialized: now 1790284353721630 us; error 28 us; skew 500 ppm
Sep 24 21:12:33 dist-test-slave-q276 krb5kdc[11022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284353, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:33.724268 11128 init.cc:377] Logged in from keytab as kudu/127.6.210.129@KRBTEST.COM (short username kudu)
I20260924 21:12:33.724608 11128 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:33.724655 11128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:33.724798 11128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:33.725823 11128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-0/data/instance:
uuid: "20e4eb7bee4f48cbb00c91beaefda53d"
format_stamp: "Formatted at 2026-09-24 21:12:33 on dist-test-slave-q276"
server_key: "cf2306d3e33f41d1abe7006857323fbb"
server_key_iv: "7a22abed8b9f624450885ad2ef7b7318"
server_key_version: "encryptionkey@0"
I20260924 21:12:33.726181 11128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance:
uuid: "20e4eb7bee4f48cbb00c91beaefda53d"
format_stamp: "Formatted at 2026-09-24 21:12:33 on dist-test-slave-q276"
server_key: "cf2306d3e33f41d1abe7006857323fbb"
server_key_iv: "7a22abed8b9f624450885ad2ef7b7318"
server_key_version: "encryptionkey@0"
I20260924 21:12:33.727448 11128 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:12:33.728157 11142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:33.728335 11128 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:12:33.728406 11128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-0/wal
uuid: "20e4eb7bee4f48cbb00c91beaefda53d"
format_stamp: "Formatted at 2026-09-24 21:12:33 on dist-test-slave-q276"
server_key: "cf2306d3e33f41d1abe7006857323fbb"
server_key_iv: "7a22abed8b9f624450885ad2ef7b7318"
server_key_version: "encryptionkey@0"
I20260924 21:12:33.728494 11128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:33.754525 11128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:33.756381 11128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:33.756528 11128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:33.756783 11128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:33.757110 11128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:33.757171 11128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:33.757206 11128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:33.757234 11128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:33.763128 11128 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.129:46425
I20260924 21:12:33.763167 11255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.129:46425 every 8 connection(s)
I20260924 21:12:33.763429 11128 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
Sep 24 21:12:33 dist-test-slave-q276 krb5kdc[11022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284353, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:33.770069 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 11128
I20260924 21:12:33.770201 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance
I20260924 21:12:33.770401 6986 external_mini_cluster.cc:1428] Setting key e5092cf9c9156bfb81cd2a427d181591
I20260924 21:12:33.772663 11124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:33.764113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:34413 (local address 127.6.210.190:37239)
0924 21:12:33.764165 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:33.764167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:33.764377 (+ 210us) server_negotiation.cc:407] Connection header received
0924 21:12:33.764813 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:33.764815 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:33.764827 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:33.764879 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:33.765753 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:33.766237 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:33.767250 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:33.767444 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:33.767493 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:33.769691 (+ 2198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:33.769692 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:33.769693 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:33.769699 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:33.771310 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:33.771713 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:33.771714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:33.771715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:33.771776 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:33.772019 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:33.772020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:33.772020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:33.772139 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:33.772218 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:33.772480 (+ 262us) server_negotiation.cc:299] Negotiation successful
0924 21:12:33.772565 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260924 21:12:33.772858 11258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:33.764105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37239 (local address 127.6.210.129:34413)
0924 21:12:33.764250 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:33.764283 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:33.764730 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:33.764933 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:33.764936 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:33.765186 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:33.765673 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:33.765678 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:33.766392 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:33.766392 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:33.767178 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:33.767179 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:33.767512 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:33.767512 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:33.767608 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:33.768167 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:33.768175 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:33.769598 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:33.771398 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:33.771404 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:33.771406 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:33.771627 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:33.771841 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:33.771842 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:33.771842 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:33.771949 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:33.772208 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:33.772214 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:33.772352 (+ 138us) client_negotiation.cc:769] Sending connection context
0924 21:12:33.772458 (+ 106us) client_negotiation.cc:240] Negotiation successful
0924 21:12:33.772612 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":89,"threads_started":1}
I20260924 21:12:33.773311 11256 heartbeater.cc:347] Connected to a master server at 127.6.210.190:37239
I20260924 21:12:33.773408 11256 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:33.773559 11256 heartbeater.cc:511] Master 127.6.210.190:37239 requested a full tablet report, sending...
I20260924 21:12:33.774035 11066 ts_manager.cc:194] Registered new tserver with Master: 20e4eb7bee4f48cbb00c91beaefda53d (127.6.210.129:46425)
I20260924 21:12:33.774698 11066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.129@KRBTEST.COM'} at 127.6.210.129:34413
WARNING: no policy specified for kudu/127.6.210.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
I20260924 21:12:33.795080 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.130:0
--local_ip_for_outbound_sockets=127.6.210.130
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:37239
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:45687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:33.876066 11261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:33.876268 11261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:33.876291 11261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:33.877378 11261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:33.877415 11261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:33.877441 11261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.130
I20260924 21:12:33.878875 11261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:45687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:37239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.130
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:33.879124 11261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:33.879305 11261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:33.884123 11266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:33.884125 11269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:33.884135 11267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:33.884338 11261 server_base.cc:1034] running on GCE node
I20260924 21:12:33.884599 11261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:33.884842 11261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:33.886013 11261 hybrid_clock.cc:648] HybridClock initialized: now 1790284353885998 us; error 46 us; skew 500 ppm
Sep 24 21:12:33 dist-test-slave-q276 krb5kdc[11022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284353, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:33.888549 11261 init.cc:377] Logged in from keytab as kudu/127.6.210.130@KRBTEST.COM (short username kudu)
I20260924 21:12:33.888885 11261 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:33.888955 11261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:33.889106 11261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:33.890118 11261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-1/data/instance:
uuid: "976806cbeef1418c9942df96d69fbfe7"
format_stamp: "Formatted at 2026-09-24 21:12:33 on dist-test-slave-q276"
server_key: "8d3ba9bae1de943fb008cb05216d155c"
server_key_iv: "9e49bc20a8bd755d5eeb26f8df155d25"
server_key_version: "encryptionkey@0"
I20260924 21:12:33.890506 11261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance:
uuid: "976806cbeef1418c9942df96d69fbfe7"
format_stamp: "Formatted at 2026-09-24 21:12:33 on dist-test-slave-q276"
server_key: "8d3ba9bae1de943fb008cb05216d155c"
server_key_iv: "9e49bc20a8bd755d5eeb26f8df155d25"
server_key_version: "encryptionkey@0"
I20260924 21:12:33.891767 11261 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:12:33.892436 11275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:33.892620 11261 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:12:33.892721 11261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-1/wal
uuid: "976806cbeef1418c9942df96d69fbfe7"
format_stamp: "Formatted at 2026-09-24 21:12:33 on dist-test-slave-q276"
server_key: "8d3ba9bae1de943fb008cb05216d155c"
server_key_iv: "9e49bc20a8bd755d5eeb26f8df155d25"
server_key_version: "encryptionkey@0"
I20260924 21:12:33.892800 11261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:33.915623 11261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:33.918083 11261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:33.918255 11261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:33.918556 11261 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:33.918883 11261 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:33.918938 11261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:33.918983 11261 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:33.919006 11261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:33.924824 11261 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.130:38369
I20260924 21:12:33.924863 11388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.130:38369 every 8 connection(s)
I20260924 21:12:33.925163 11261 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
I20260924 21:12:33.929795 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 11261
I20260924 21:12:33.930169 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance
I20260924 21:12:33.930413 6986 external_mini_cluster.cc:1428] Setting key a7118390cbf4be159a22e12f0b473f76
Sep 24 21:12:33 dist-test-slave-q276 krb5kdc[11022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284353, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:33.935849 11124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:33.925970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:44299 (local address 127.6.210.190:37239)
0924 21:12:33.926047 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:33.926049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:33.926116 (+ 67us) server_negotiation.cc:407] Connection header received
0924 21:12:33.926685 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:33.926686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:33.926695 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:33.926760 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:33.927701 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:33.928209 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:33.929241 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:33.929450 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:33.929540 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:33.932731 (+ 3191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:33.932733 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:33.932733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:33.932741 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:33.934432 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:33.934831 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:33.934833 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:33.934833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:33.934897 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:33.935171 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:33.935172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:33.935172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:33.935318 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:33.935394 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:33.935636 (+ 242us) server_negotiation.cc:299] Negotiation successful
0924 21:12:33.935730 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260924 21:12:33.935873 11391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:33.925840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37239 (local address 127.6.210.130:44299)
0924 21:12:33.925992 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:33.926034 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:33.926591 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:33.926837 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:33.926840 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:33.927095 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:33.927597 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:33.927602 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:33.928289 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:33.928290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:33.929099 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:33.929101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:33.929592 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:33.929594 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:33.929746 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:33.930522 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:33.930534 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:33.932582 (+ 2048us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:33.934523 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:33.934528 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:33.934531 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:33.934753 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:33.934976 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:33.934977 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:33.934978 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:33.935098 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:33.935386 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:33.935390 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:33.935506 (+ 116us) client_negotiation.cc:769] Sending connection context
0924 21:12:33.935589 (+ 83us) client_negotiation.cc:240] Negotiation successful
0924 21:12:33.935713 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
I20260924 21:12:33.936376 11389 heartbeater.cc:347] Connected to a master server at 127.6.210.190:37239
I20260924 21:12:33.936488 11389 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:33.936662 11389 heartbeater.cc:511] Master 127.6.210.190:37239 requested a full tablet report, sending...
I20260924 21:12:33.937058 11066 ts_manager.cc:194] Registered new tserver with Master: 976806cbeef1418c9942df96d69fbfe7 (127.6.210.130:38369)
I20260924 21:12:33.937562 11066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.130@KRBTEST.COM'} at 127.6.210.130:44299
WARNING: no policy specified for kudu/127.6.210.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
I20260924 21:12:33.957261 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.131:0
--local_ip_for_outbound_sockets=127.6.210.131
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:37239
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:45687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:34.037706 11394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:34.037951 11394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:34.037989 11394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:34.039208 11394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:34.039278 11394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:34.039323 11394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.131
I20260924 21:12:34.040877 11394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:45687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:37239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.131
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:34.041145 11394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:34.041380 11394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:34.046408 11399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:34.046382 11400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:34.046422 11394 server_base.cc:1034] running on GCE node
W20260924 21:12:34.046384 11402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:34.046846 11394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:34.047134 11394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:34.048295 11394 hybrid_clock.cc:648] HybridClock initialized: now 1790284354048264 us; error 46 us; skew 500 ppm
Sep 24 21:12:34 dist-test-slave-q276 krb5kdc[11022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284354, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:34.050720 11394 init.cc:377] Logged in from keytab as kudu/127.6.210.131@KRBTEST.COM (short username kudu)
I20260924 21:12:34.051110 11394 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:34.051183 11394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:34.051339 11394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:34.052371 11394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-2/data/instance:
uuid: "aa35a15b58514ddb99cfc93598e9f1d5"
format_stamp: "Formatted at 2026-09-24 21:12:34 on dist-test-slave-q276"
server_key: "1afe0de8f8ea62829645da01f49472f9"
server_key_iv: "02feea374a215e83e335553b35f25f1d"
server_key_version: "encryptionkey@0"
I20260924 21:12:34.052738 11394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance:
uuid: "aa35a15b58514ddb99cfc93598e9f1d5"
format_stamp: "Formatted at 2026-09-24 21:12:34 on dist-test-slave-q276"
server_key: "1afe0de8f8ea62829645da01f49472f9"
server_key_iv: "02feea374a215e83e335553b35f25f1d"
server_key_version: "encryptionkey@0"
I20260924 21:12:34.053974 11394 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:12:34.054719 11408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:34.054883 11394 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:34.054986 11394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-2/wal
uuid: "aa35a15b58514ddb99cfc93598e9f1d5"
format_stamp: "Formatted at 2026-09-24 21:12:34 on dist-test-slave-q276"
server_key: "1afe0de8f8ea62829645da01f49472f9"
server_key_iv: "02feea374a215e83e335553b35f25f1d"
server_key_version: "encryptionkey@0"
I20260924 21:12:34.055076 11394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:34.070076 11394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:34.072053 11394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:34.072229 11394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:34.072458 11394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:34.072765 11394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:34.072816 11394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:34.072857 11394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:34.072886 11394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:34.078686 11394 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.131:46783
I20260924 21:12:34.078727 11521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.131:46783 every 8 connection(s)
I20260924 21:12:34.078984 11394 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
I20260924 21:12:34.082006 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 11394
I20260924 21:12:34.082140 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance
I20260924 21:12:34.082363 6986 external_mini_cluster.cc:1428] Setting key 30d427c2d2c048a8bc6ff02bdebe58d3
Sep 24 21:12:34 dist-test-slave-q276 krb5kdc[11022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284354, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:34.088619 11124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.079661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:48833 (local address 127.6.210.190:37239)
0924 21:12:34.079736 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:34.079738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:34.079941 (+ 203us) server_negotiation.cc:407] Connection header received
0924 21:12:34.080390 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:34.080392 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:34.080402 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:34.080475 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:34.081438 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:34.081979 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:34.083450 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:34.083638 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:34.083693 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:34.085676 (+ 1983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:34.085677 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:34.085677 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:34.085684 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:34.087267 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:34.087657 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:34.087658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:34.087658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:34.087708 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:34.087964 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:34.087966 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:34.087966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:34.088087 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:34.088161 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:34.088484 (+ 323us) server_negotiation.cc:299] Negotiation successful
0924 21:12:34.088554 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 21:12:34.088847 11524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.079639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37239 (local address 127.6.210.131:48833)
0924 21:12:34.079821 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:34.079858 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:34.080311 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:34.080562 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:34.080567 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:34.080858 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:34.081358 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:34.081363 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:34.082089 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:34.082090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:34.083385 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:34.083387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:34.083709 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:34.083710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:34.083803 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:34.084242 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:34.084250 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:34.085591 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:34.087338 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:34.087345 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:34.087346 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:34.087575 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:34.087765 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:34.087766 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:34.087766 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:34.087875 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:34.088207 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:34.088214 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:34.088334 (+ 120us) client_negotiation.cc:769] Sending connection context
0924 21:12:34.088451 (+ 117us) client_negotiation.cc:240] Negotiation successful
0924 21:12:34.088582 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":102,"threads_started":1}
I20260924 21:12:34.089468 11522 heartbeater.cc:347] Connected to a master server at 127.6.210.190:37239
I20260924 21:12:34.089596 11522 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:34.089768 11522 heartbeater.cc:511] Master 127.6.210.190:37239 requested a full tablet report, sending...
I20260924 21:12:34.090107 11066 ts_manager.cc:194] Registered new tserver with Master: aa35a15b58514ddb99cfc93598e9f1d5 (127.6.210.131:46783)
I20260924 21:12:34.090552 11066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.131@KRBTEST.COM'} at 127.6.210.131:48833
I20260924 21:12:34.094010 6986 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:12:34.101080 11124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.095391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42250 (local address 127.6.210.190:37239)
0924 21:12:34.095443 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:34.095445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:34.095555 (+ 110us) server_negotiation.cc:407] Connection header received
0924 21:12:34.095611 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:34.095612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:34.095619 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:34.095666 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:34.096200 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:34.096748 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:34.097436 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:34.097625 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:34.097676 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:34.098815 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:34.098816 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:34.098817 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:34.098822 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:34.100201 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:34.100482 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:34.100483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:34.100483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:34.100531 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:34.100694 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:34.100694 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:34.100694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:34.100814 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:34.100881 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:34.100966 (+ 85us) server_negotiation.cc:299] Negotiation successful
0924 21:12:34.101018 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 21:12:34.102761 11066 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42250:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:12:34.104235 11066 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:12:34.111016 11536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.106881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:46425 (local address 127.0.0.1:42488)
0924 21:12:34.107276 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:34.107286 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:34.107335 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:34.107649 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:34.107650 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:34.107652 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:34.107709 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:34.107710 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:34.108876 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:34.108877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:34.109784 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:34.109786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:34.110706 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:34.110707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:34.110780 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:34.110795 (+ 15us) client_negotiation.cc:769] Sending connection context
0924 21:12:34.110874 (+ 79us) client_negotiation.cc:240] Negotiation successful
0924 21:12:34.110927 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":5,"thread_start_us":93,"threads_started":1}
I20260924 21:12:34.111027 11538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.107103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42488 (local address 127.6.210.129:46425)
0924 21:12:34.107420 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:34.107424 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:34.107435 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:12:34.107476 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:34.107478 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:34.107495 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:34.107567 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:34.107778 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:34.108785 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:34.109868 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:34.110617 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:34.110697 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:34.110760 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:34.110897 (+ 137us) server_negotiation.cc:299] Negotiation successful
0924 21:12:34.110944 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":91,"threads_started":1}
I20260924 21:12:34.111186 11535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.106776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39994 (local address 127.6.210.130:38369)
0924 21:12:34.107104 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:34.107109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:34.107119 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:12:34.107159 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:34.107161 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:34.107172 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:34.107263 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:34.107441 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:34.108593 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:34.109980 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:34.110692 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:34.110750 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:34.110795 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:34.111058 (+ 263us) server_negotiation.cc:299] Negotiation successful
0924 21:12:34.111105 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":71,"threads_started":1}
I20260924 21:12:34.111344 11534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.106684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:38369 (local address 127.0.0.1:39994)
0924 21:12:34.106867 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:34.106912 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:34.106978 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:34.107267 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:34.107271 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:34.107273 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:34.107354 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:34.107356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:34.108687 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:34.108688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:34.109896 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:34.109899 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:34.110834 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:34.110835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:34.110913 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:34.110917 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:12:34.110967 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 21:12:34.111006 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":28,"thread_start_us":60,"threads_started":1}
I20260924 21:12:34.111783 11533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.106721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:46783 (local address 127.0.0.1:42364)
0924 21:12:34.106928 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:34.106938 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:34.106991 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:34.107614 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:34.107615 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:34.107617 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:34.107653 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:34.107654 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:34.108528 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:34.108529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:34.109897 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:34.109899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:34.111069 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:34.111070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:34.111612 (+ 542us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:34.111617 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 21:12:34.111674 (+ 57us) client_negotiation.cc:240] Negotiation successful
0924 21:12:34.111712 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":46,"thread_start_us":94,"threads_started":1}
I20260924 21:12:34.111778 11190 tablet_service.cc:1467] Processing CreateTablet for tablet 69ca020437ac42f0b61502f26b2e26dd (DEFAULT_TABLE table=test-table [id=e2210f6fe616453e98a0d69990966925]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:34.112082 11537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.106884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42364 (local address 127.6.210.131:46783)
0924 21:12:34.107391 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:34.107396 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:34.107406 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:12:34.107446 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:34.107448 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:34.107462 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:34.107535 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:34.107734 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:34.108415 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:34.110336 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:34.110927 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:34.110988 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:34.111036 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:34.111962 (+ 926us) server_negotiation.cc:299] Negotiation successful
0924 21:12:34.112006 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":89,"threads_started":1}
I20260924 21:12:34.112147 11190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69ca020437ac42f0b61502f26b2e26dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:34.112062 11323 tablet_service.cc:1467] Processing CreateTablet for tablet 69ca020437ac42f0b61502f26b2e26dd (DEFAULT_TABLE table=test-table [id=e2210f6fe616453e98a0d69990966925]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:34.112406 11323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69ca020437ac42f0b61502f26b2e26dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:34.112643 11456 tablet_service.cc:1467] Processing CreateTablet for tablet 69ca020437ac42f0b61502f26b2e26dd (DEFAULT_TABLE table=test-table [id=e2210f6fe616453e98a0d69990966925]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:34.112944 11456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69ca020437ac42f0b61502f26b2e26dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:34.115128 11539 tablet_bootstrap.cc:492] T 69ca020437ac42f0b61502f26b2e26dd P 976806cbeef1418c9942df96d69fbfe7: Bootstrap starting.
I20260924 21:12:34.115531 11540 tablet_bootstrap.cc:492] T 69ca020437ac42f0b61502f26b2e26dd P 20e4eb7bee4f48cbb00c91beaefda53d: Bootstrap starting.
I20260924 21:12:34.115950 11539 tablet_bootstrap.cc:654] T 69ca020437ac42f0b61502f26b2e26dd P 976806cbeef1418c9942df96d69fbfe7: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:34.116112 11541 tablet_bootstrap.cc:492] T 69ca020437ac42f0b61502f26b2e26dd P aa35a15b58514ddb99cfc93598e9f1d5: Bootstrap starting.
I20260924 21:12:34.116230 11540 tablet_bootstrap.cc:654] T 69ca020437ac42f0b61502f26b2e26dd P 20e4eb7bee4f48cbb00c91beaefda53d: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:34.116779 11540 log.cc:824] T 69ca020437ac42f0b61502f26b2e26dd P 20e4eb7bee4f48cbb00c91beaefda53d: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:34.116792 11541 tablet_bootstrap.cc:654] T 69ca020437ac42f0b61502f26b2e26dd P aa35a15b58514ddb99cfc93598e9f1d5: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:34.116770 11539 log.cc:824] T 69ca020437ac42f0b61502f26b2e26dd P 976806cbeef1418c9942df96d69fbfe7: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:34.117337 11541 log.cc:824] T 69ca020437ac42f0b61502f26b2e26dd P aa35a15b58514ddb99cfc93598e9f1d5: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:34.117563 11540 tablet_bootstrap.cc:492] T 69ca020437ac42f0b61502f26b2e26dd P 20e4eb7bee4f48cbb00c91beaefda53d: No bootstrap required, opened a new log
I20260924 21:12:34.117630 11540 ts_tablet_manager.cc:1397] T 69ca020437ac42f0b61502f26b2e26dd P 20e4eb7bee4f48cbb00c91beaefda53d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:12:34.118062 11541 tablet_bootstrap.cc:492] T 69ca020437ac42f0b61502f26b2e26dd P aa35a15b58514ddb99cfc93598e9f1d5: No bootstrap required, opened a new log
I20260924 21:12:34.118134 11541 ts_tablet_manager.cc:1397] T 69ca020437ac42f0b61502f26b2e26dd P aa35a15b58514ddb99cfc93598e9f1d5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:12:34.118698 11539 tablet_bootstrap.cc:492] T 69ca020437ac42f0b61502f26b2e26dd P 976806cbeef1418c9942df96d69fbfe7: No bootstrap required, opened a new log
I20260924 21:12:34.118781 11539 ts_tablet_manager.cc:1397] T 69ca020437ac42f0b61502f26b2e26dd P 976806cbeef1418c9942df96d69fbfe7: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260924 21:12:34.119652 11540 raft_consensus.cc:348] T 69ca020437ac42f0b61502f26b2e26dd P 20e4eb7bee4f48cbb00c91beaefda53d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "976806cbeef1418c9942df96d69fbfe7" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38369 } } peers { permanent_uuid: "20e4eb7bee4f48cbb00c91beaefda53d" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 46425 } } peers { permanent_uuid: "aa35a15b58514ddb99cfc93598e9f1d5" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46783 } }
I20260924 21:12:34.119835 11540 raft_consensus.cc:374] T 69ca020437ac42f0b61502f26b2e26dd P 20e4eb7bee4f48cbb00c91beaefda53d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:34.119869 11540 raft_consensus.cc:729] T 69ca020437ac42f0b61502f26b2e26dd P 20e4eb7bee4f48cbb00c91beaefda53d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20e4eb7bee4f48cbb00c91beaefda53d, State: Initialized, Role: FOLLOWER
I20260924 21:12:34.119984 11540 consensus_queue.cc:260] T 69ca020437ac42f0b61502f26b2e26dd P 20e4eb7bee4f48cbb00c91beaefda53d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "976806cbeef1418c9942df96d69fbfe7" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38369 } } peers { permanent_uuid: "20e4eb7bee4f48cbb00c91beaefda53d" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 46425 } } peers { permanent_uuid: "aa35a15b58514ddb99cfc93598e9f1d5" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46783 } }
I20260924 21:12:34.120021 11541 raft_consensus.cc:348] T 69ca020437ac42f0b61502f26b2e26dd P aa35a15b58514ddb99cfc93598e9f1d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "976806cbeef1418c9942df96d69fbfe7" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38369 } } peers { permanent_uuid: "20e4eb7bee4f48cbb00c91beaefda53d" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 46425 } } peers { permanent_uuid: "aa35a15b58514ddb99cfc93598e9f1d5" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46783 } }
I20260924 21:12:34.120177 11541 raft_consensus.cc:374] T 69ca020437ac42f0b61502f26b2e26dd P aa35a15b58514ddb99cfc93598e9f1d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:34.120221 11541 raft_consensus.cc:729] T 69ca020437ac42f0b61502f26b2e26dd P aa35a15b58514ddb99cfc93598e9f1d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa35a15b58514ddb99cfc93598e9f1d5, State: Initialized, Role: FOLLOWER
I20260924 21:12:34.120335 11540 ts_tablet_manager.cc:1428] T 69ca020437ac42f0b61502f26b2e26dd P 20e4eb7bee4f48cbb00c91beaefda53d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:12:34.120333 11541 consensus_queue.cc:260] T 69ca020437ac42f0b61502f26b2e26dd P aa35a15b58514ddb99cfc93598e9f1d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "976806cbeef1418c9942df96d69fbfe7" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38369 } } peers { permanent_uuid: "20e4eb7bee4f48cbb00c91beaefda53d" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 46425 } } peers { permanent_uuid: "aa35a15b58514ddb99cfc93598e9f1d5" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46783 } }
I20260924 21:12:34.120376 11256 heartbeater.cc:503] Master 127.6.210.190:37239 was elected leader, sending a full tablet report...
I20260924 21:12:34.120267 11539 raft_consensus.cc:348] T 69ca020437ac42f0b61502f26b2e26dd P 976806cbeef1418c9942df96d69fbfe7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "976806cbeef1418c9942df96d69fbfe7" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38369 } } peers { permanent_uuid: "20e4eb7bee4f48cbb00c91beaefda53d" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 46425 } } peers { permanent_uuid: "aa35a15b58514ddb99cfc93598e9f1d5" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46783 } }
I20260924 21:12:34.120416 11539 raft_consensus.cc:374] T 69ca020437ac42f0b61502f26b2e26dd P 976806cbeef1418c9942df96d69fbfe7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:34.120447 11539 raft_consensus.cc:729] T 69ca020437ac42f0b61502f26b2e26dd P 976806cbeef1418c9942df96d69fbfe7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 976806cbeef1418c9942df96d69fbfe7, State: Initialized, Role: FOLLOWER
I20260924 21:12:34.120590 11522 heartbeater.cc:503] Master 127.6.210.190:37239 was elected leader, sending a full tablet report...
I20260924 21:12:34.120576 11539 consensus_queue.cc:260] T 69ca020437ac42f0b61502f26b2e26dd P 976806cbeef1418c9942df96d69fbfe7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "976806cbeef1418c9942df96d69fbfe7" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38369 } } peers { permanent_uuid: "20e4eb7bee4f48cbb00c91beaefda53d" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 46425 } } peers { permanent_uuid: "aa35a15b58514ddb99cfc93598e9f1d5" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46783 } }
I20260924 21:12:34.120589 11541 ts_tablet_manager.cc:1428] T 69ca020437ac42f0b61502f26b2e26dd P aa35a15b58514ddb99cfc93598e9f1d5: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 21:12:34.120849 11389 heartbeater.cc:503] Master 127.6.210.190:37239 was elected leader, sending a full tablet report...
I20260924 21:12:34.120859 11539 ts_tablet_manager.cc:1428] T 69ca020437ac42f0b61502f26b2e26dd P 976806cbeef1418c9942df96d69fbfe7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260924 21:12:34.175761 11390 tablet.cc:2369] T 69ca020437ac42f0b61502f26b2e26dd P 976806cbeef1418c9942df96d69fbfe7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 21:12:34.264122 11257 tablet.cc:2369] T 69ca020437ac42f0b61502f26b2e26dd P 20e4eb7bee4f48cbb00c91beaefda53d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 21:12:34.329488 11523 tablet.cc:2369] T 69ca020437ac42f0b61502f26b2e26dd P aa35a15b58514ddb99cfc93598e9f1d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:12:34.346792 11545 raft_consensus.cc:482] T 69ca020437ac42f0b61502f26b2e26dd P 20e4eb7bee4f48cbb00c91beaefda53d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:12:34.346916 11545 raft_consensus.cc:504] T 69ca020437ac42f0b61502f26b2e26dd P 20e4eb7bee4f48cbb00c91beaefda53d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "976806cbeef1418c9942df96d69fbfe7" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38369 } } peers { permanent_uuid: "20e4eb7bee4f48cbb00c91beaefda53d" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 46425 } } peers { permanent_uuid: "aa35a15b58514ddb99cfc93598e9f1d5" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46783 } }
I20260924 21:12:34.347299 11545 leader_election.cc:288] T 69ca020437ac42f0b61502f26b2e26dd P 20e4eb7bee4f48cbb00c91beaefda53d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 976806cbeef1418c9942df96d69fbfe7 (127.6.210.130:38369), aa35a15b58514ddb99cfc93598e9f1d5 (127.6.210.131:46783)
I20260924 21:12:34.350234 11535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.347546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:33299 (local address 127.6.210.130:38369)
0924 21:12:34.347611 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:34.347613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:34.347733 (+ 120us) server_negotiation.cc:407] Connection header received
0924 21:12:34.347805 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:34.347806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:34.347817 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:34.347883 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:34.348058 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:34.348523 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:34.349494 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:34.349864 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:34.349949 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:34.349957 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:34.350125 (+ 168us) server_negotiation.cc:299] Negotiation successful
0924 21:12:34.350160 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 21:12:34.350235 11537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.347567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:50659 (local address 127.6.210.131:46783)
0924 21:12:34.347619 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:34.347621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:34.347688 (+ 67us) server_negotiation.cc:407] Connection header received
0924 21:12:34.347762 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:34.347763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:34.347775 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:34.347825 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:34.347998 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:34.348518 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:34.349493 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:34.349868 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:34.349953 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:34.349961 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:34.350116 (+ 155us) server_negotiation.cc:299] Negotiation successful
0924 21:12:34.350160 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 21:12:34.350294 11548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.347470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:46783 (local address 127.6.210.129:50659)
0924 21:12:34.347604 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:34.347616 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:34.347706 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:34.347827 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:34.347828 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:34.347830 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:34.347902 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:34.347904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:34.348605 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:34.348606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:34.349397 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:34.349399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:34.349953 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:34.349954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:34.350033 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:34.350036 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:12:34.350111 (+ 75us) client_negotiation.cc:240] Negotiation successful
0924 21:12:34.350153 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":2,"spinlock_wait_cycles":15616,"thread_start_us":77,"threads_started":1}
I20260924 21:12:34.350641 11476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69ca020437ac42f0b61502f26b2e26dd" candidate_uuid: "20e4eb7bee4f48cbb00c91beaefda53d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa35a15b58514ddb99cfc93598e9f1d5" is_pre_election: true
I20260924 21:12:34.350299 11549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.347471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:38369 (local address 127.6.210.129:33299)
0924 21:12:34.347638 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:34.347649 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:34.347744 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:34.347898 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:34.347899 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:34.347901 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:34.347986 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:34.347988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:34.348625 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:34.348626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:34.349395 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:34.349397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:34.349948 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:34.349948 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:34.350027 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:34.350030 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:12:34.350110 (+ 80us) client_negotiation.cc:240] Negotiation successful
0924 21:12:34.350153 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":9,"thread_start_us":64,"threads_started":1}
I20260924 21:12:34.350800 11476 raft_consensus.cc:2454] T 69ca020437ac42f0b61502f26b2e26dd P aa35a15b58514ddb99cfc93598e9f1d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20e4eb7bee4f48cbb00c91beaefda53d in term 0.
I20260924 21:12:34.351028 11143 leader_election.cc:302] T 69ca020437ac42f0b61502f26b2e26dd P 20e4eb7bee4f48cbb00c91beaefda53d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20e4eb7bee4f48cbb00c91beaefda53d, aa35a15b58514ddb99cfc93598e9f1d5; no voters:
I20260924 21:12:34.351171 11545 raft_consensus.cc:2781] T 69ca020437ac42f0b61502f26b2e26dd P 20e4eb7bee4f48cbb00c91beaefda53d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:12:34.351168 11343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69ca020437ac42f0b61502f26b2e26dd" candidate_uuid: "20e4eb7bee4f48cbb00c91beaefda53d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "976806cbeef1418c9942df96d69fbfe7" is_pre_election: true
I20260924 21:12:34.351239 11545 raft_consensus.cc:482] T 69ca020437ac42f0b61502f26b2e26dd P 20e4eb7bee4f48cbb00c91beaefda53d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:12:34.351266 11545 raft_consensus.cc:3037] T 69ca020437ac42f0b61502f26b2e26dd P 20e4eb7bee4f48cbb00c91beaefda53d [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:34.351305 11343 raft_consensus.cc:2454] T 69ca020437ac42f0b61502f26b2e26dd P 976806cbeef1418c9942df96d69fbfe7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20e4eb7bee4f48cbb00c91beaefda53d in term 0.
I20260924 21:12:34.352011 11545 raft_consensus.cc:504] T 69ca020437ac42f0b61502f26b2e26dd P 20e4eb7bee4f48cbb00c91beaefda53d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "976806cbeef1418c9942df96d69fbfe7" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38369 } } peers { permanent_uuid: "20e4eb7bee4f48cbb00c91beaefda53d" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 46425 } } peers { permanent_uuid: "aa35a15b58514ddb99cfc93598e9f1d5" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46783 } }
I20260924 21:12:34.352164 11545 leader_election.cc:288] T 69ca020437ac42f0b61502f26b2e26dd P 20e4eb7bee4f48cbb00c91beaefda53d [CANDIDATE]: Term 1 election: Requested vote from peers 976806cbeef1418c9942df96d69fbfe7 (127.6.210.130:38369), aa35a15b58514ddb99cfc93598e9f1d5 (127.6.210.131:46783)
I20260924 21:12:34.352308 11476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69ca020437ac42f0b61502f26b2e26dd" candidate_uuid: "20e4eb7bee4f48cbb00c91beaefda53d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa35a15b58514ddb99cfc93598e9f1d5"
I20260924 21:12:34.352306 11343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69ca020437ac42f0b61502f26b2e26dd" candidate_uuid: "20e4eb7bee4f48cbb00c91beaefda53d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "976806cbeef1418c9942df96d69fbfe7"
I20260924 21:12:34.352381 11343 raft_consensus.cc:3037] T 69ca020437ac42f0b61502f26b2e26dd P 976806cbeef1418c9942df96d69fbfe7 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:34.352381 11476 raft_consensus.cc:3037] T 69ca020437ac42f0b61502f26b2e26dd P aa35a15b58514ddb99cfc93598e9f1d5 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:34.353262 11476 raft_consensus.cc:2454] T 69ca020437ac42f0b61502f26b2e26dd P aa35a15b58514ddb99cfc93598e9f1d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20e4eb7bee4f48cbb00c91beaefda53d in term 1.
I20260924 21:12:34.353262 11343 raft_consensus.cc:2454] T 69ca020437ac42f0b61502f26b2e26dd P 976806cbeef1418c9942df96d69fbfe7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20e4eb7bee4f48cbb00c91beaefda53d in term 1.
I20260924 21:12:34.353435 11143 leader_election.cc:302] T 69ca020437ac42f0b61502f26b2e26dd P 20e4eb7bee4f48cbb00c91beaefda53d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20e4eb7bee4f48cbb00c91beaefda53d, aa35a15b58514ddb99cfc93598e9f1d5; no voters:
I20260924 21:12:34.353533 11545 raft_consensus.cc:2781] T 69ca020437ac42f0b61502f26b2e26dd P 20e4eb7bee4f48cbb00c91beaefda53d [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:34.353703 11545 raft_consensus.cc:686] T 69ca020437ac42f0b61502f26b2e26dd P 20e4eb7bee4f48cbb00c91beaefda53d [term 1 LEADER]: Becoming Leader. State: Replica: 20e4eb7bee4f48cbb00c91beaefda53d, State: Running, Role: LEADER
I20260924 21:12:34.353811 11545 consensus_queue.cc:237] T 69ca020437ac42f0b61502f26b2e26dd P 20e4eb7bee4f48cbb00c91beaefda53d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "976806cbeef1418c9942df96d69fbfe7" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38369 } } peers { permanent_uuid: "20e4eb7bee4f48cbb00c91beaefda53d" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 46425 } } peers { permanent_uuid: "aa35a15b58514ddb99cfc93598e9f1d5" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46783 } }
I20260924 21:12:34.354679 11066 catalog_manager.cc:5495] T 69ca020437ac42f0b61502f26b2e26dd P 20e4eb7bee4f48cbb00c91beaefda53d reported cstate change: term changed from 0 to 1, leader changed from <none> to 20e4eb7bee4f48cbb00c91beaefda53d (127.6.210.129). New cstate: current_term: 1 leader_uuid: "20e4eb7bee4f48cbb00c91beaefda53d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "976806cbeef1418c9942df96d69fbfe7" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20e4eb7bee4f48cbb00c91beaefda53d" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 46425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa35a15b58514ddb99cfc93598e9f1d5" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46783 } health_report { overall_health: UNKNOWN } } }
I20260924 21:12:34.392952 11538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.390736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42492 (local address 127.6.210.129:46425)
0924 21:12:34.390797 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:34.390799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:34.390806 (+ 7us) server_negotiation.cc:407] Connection header received
0924 21:12:34.390839 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:34.390840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:34.390858 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:34.390925 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:12:34.391074 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:34.391533 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:34.392260 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:34.392508 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:34.392559 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:34.392691 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:12:34.392788 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:12:34.392848 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:34.392880 (+ 32us) server_negotiation.cc:299] Negotiation successful
0924 21:12:34.392896 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 21:12:34.394807 11476 raft_consensus.cc:1261] T 69ca020437ac42f0b61502f26b2e26dd P aa35a15b58514ddb99cfc93598e9f1d5 [term 1 FOLLOWER]: Refusing update from remote peer 20e4eb7bee4f48cbb00c91beaefda53d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:34.394845 11343 raft_consensus.cc:1261] T 69ca020437ac42f0b61502f26b2e26dd P 976806cbeef1418c9942df96d69fbfe7 [term 1 FOLLOWER]: Refusing update from remote peer 20e4eb7bee4f48cbb00c91beaefda53d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:34.395197 11550 consensus_queue.cc:1035] T 69ca020437ac42f0b61502f26b2e26dd P 20e4eb7bee4f48cbb00c91beaefda53d [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa35a15b58514ddb99cfc93598e9f1d5" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:12:34.395385 11545 consensus_queue.cc:1035] T 69ca020437ac42f0b61502f26b2e26dd P 20e4eb7bee4f48cbb00c91beaefda53d [LEADER]: Connected to new peer: Peer: permanent_uuid: "976806cbeef1418c9942df96d69fbfe7" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:12:34.397854 11555 mvcc.cc:205] Tried to move back new op lower bound from 7333004715598159872 to 7333004715433832448. Current Snapshot: MvccSnapshot[applied={T|T < 7333004715598159872}]
I20260924 21:12:34.398072 11553 mvcc.cc:205] Tried to move back new op lower bound from 7333004715598159872 to 7333004715433832448. Current Snapshot: MvccSnapshot[applied={T|T < 7333004715598159872}]
I20260924 21:12:34.400648 11066 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:12:34.400743 11066 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:12:34.401433 11066 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c507efc77aa443b92001d330bfa5953: Sending DeleteTablet for 3 replicas of tablet 69ca020437ac42f0b61502f26b2e26dd
I20260924 21:12:34.401737 11323 tablet_service.cc:1514] Processing DeleteTablet for tablet 69ca020437ac42f0b61502f26b2e26dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:34 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:39994
I20260924 21:12:34.401777 11456 tablet_service.cc:1514] Processing DeleteTablet for tablet 69ca020437ac42f0b61502f26b2e26dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:34 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:42364
I20260924 21:12:34.401808 11190 tablet_service.cc:1514] Processing DeleteTablet for tablet 69ca020437ac42f0b61502f26b2e26dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:34 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:42488
I20260924 21:12:34.401958 11565 tablet_replica.cc:330] T 69ca020437ac42f0b61502f26b2e26dd P aa35a15b58514ddb99cfc93598e9f1d5: stopping tablet replica
I20260924 21:12:34.402052 11565 raft_consensus.cc:2229] T 69ca020437ac42f0b61502f26b2e26dd P aa35a15b58514ddb99cfc93598e9f1d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:12:34.402123 11564 tablet_replica.cc:330] T 69ca020437ac42f0b61502f26b2e26dd P 976806cbeef1418c9942df96d69fbfe7: stopping tablet replica
I20260924 21:12:34.402174 11565 raft_consensus.cc:2258] T 69ca020437ac42f0b61502f26b2e26dd P aa35a15b58514ddb99cfc93598e9f1d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:34.402235 11564 raft_consensus.cc:2229] T 69ca020437ac42f0b61502f26b2e26dd P 976806cbeef1418c9942df96d69fbfe7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:12:34.402452 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 11128
I20260924 21:12:34.403059 11564 raft_consensus.cc:2258] T 69ca020437ac42f0b61502f26b2e26dd P 976806cbeef1418c9942df96d69fbfe7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:34.403084 11565 ts_tablet_manager.cc:1905] T 69ca020437ac42f0b61502f26b2e26dd P aa35a15b58514ddb99cfc93598e9f1d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:34.403549 11564 ts_tablet_manager.cc:1905] T 69ca020437ac42f0b61502f26b2e26dd P 976806cbeef1418c9942df96d69fbfe7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:34.404898 11565 ts_tablet_manager.cc:1918] T 69ca020437ac42f0b61502f26b2e26dd P aa35a15b58514ddb99cfc93598e9f1d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:12:34.404978 11565 log.cc:1196] T 69ca020437ac42f0b61502f26b2e26dd P aa35a15b58514ddb99cfc93598e9f1d5: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-2/wal/wals/69ca020437ac42f0b61502f26b2e26dd
I20260924 21:12:34.405256 11564 ts_tablet_manager.cc:1918] T 69ca020437ac42f0b61502f26b2e26dd P 976806cbeef1418c9942df96d69fbfe7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:12:34.405259 11565 ts_tablet_manager.cc:1939] T 69ca020437ac42f0b61502f26b2e26dd P aa35a15b58514ddb99cfc93598e9f1d5: Deleting consensus metadata
I20260924 21:12:34.405345 11564 log.cc:1196] T 69ca020437ac42f0b61502f26b2e26dd P 976806cbeef1418c9942df96d69fbfe7: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284333163253-6986-0/minicluster-data/ts-1/wal/wals/69ca020437ac42f0b61502f26b2e26dd
I20260924 21:12:34.405571 11564 ts_tablet_manager.cc:1939] T 69ca020437ac42f0b61502f26b2e26dd P 976806cbeef1418c9942df96d69fbfe7: Deleting consensus metadata
I20260924 21:12:34.405714 11050 catalog_manager.cc:4841] TS aa35a15b58514ddb99cfc93598e9f1d5 (127.6.210.131:46783): tablet 69ca020437ac42f0b61502f26b2e26dd (table test-table [id=e2210f6fe616453e98a0d69990966925]) successfully deleted
I20260924 21:12:34.406008 11053 catalog_manager.cc:4841] TS 976806cbeef1418c9942df96d69fbfe7 (127.6.210.130:38369): tablet 69ca020437ac42f0b61502f26b2e26dd (table test-table [id=e2210f6fe616453e98a0d69990966925]) successfully deleted
W20260924 21:12:34.408447 11053 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.210.129:46425 (error 108)
I20260924 21:12:34.408634 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 11261
I20260924 21:12:34.409474 11534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.409323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:46425 (local address 127.0.0.1:42506)
0924 21:12:34.409389 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:34.409432 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.129:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260924 21:12:34.410210 11053 catalog_manager.cc:4584] TS 20e4eb7bee4f48cbb00c91beaefda53d (127.6.210.129:46425): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 69ca020437ac42f0b61502f26b2e26dd: Network error: Client connection negotiation failed: client connection to 127.6.210.129:46425: connect: Connection refused (error 111)
I20260924 21:12:34.412720 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 11394
I20260924 21:12:34.417343 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 11035
2026-09-24T21:12:34Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2194 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-24T21:12:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:12:34Z Disabled control of system clock
I20260924 21:12:34.444521 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:43743
--webserver_interface=127.6.210.190
--webserver_port=0
--builtin_ntp_servers=127.6.210.148:44879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:43743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260924 21:12:34.523676 11571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:34.523885 11571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:34.523906 11571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:34.524986 11571 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:12:34.525020 11571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:34.525033 11571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:34.525046 11571 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:12:34.525058 11571 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:12:34.526371 11571 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:44879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:43743
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:43743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:34.526567 11571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:34.526791 11571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:34.531513 11577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:34.531610 11579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:34.531471 11576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:34.531723 11571 server_base.cc:1034] running on GCE node
I20260924 21:12:34.531965 11571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:34.532258 11571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:34.533419 11571 hybrid_clock.cc:648] HybridClock initialized: now 1790284354533407 us; error 30 us; skew 500 ppm
I20260924 21:12:34.534718 11571 webserver.cc:460] Webserver started at http://127.6.210.190:41347/ using document root <none> and password file <none>
I20260924 21:12:34.534971 11571 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:34.535043 11571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:34.535220 11571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:34.536350 11571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "2381242e210942f3ac854ebd44212430"
format_stamp: "Formatted at 2026-09-24 21:12:34 on dist-test-slave-q276"
server_key: "8f286b3c92e973a76bf7574c04dcb66b"
server_key_iv: "ab7aed87e4a783fc4896a81ffdf7c609"
server_key_version: "encryptionkey@0"
I20260924 21:12:34.536762 11571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "2381242e210942f3ac854ebd44212430"
format_stamp: "Formatted at 2026-09-24 21:12:34 on dist-test-slave-q276"
server_key: "8f286b3c92e973a76bf7574c04dcb66b"
server_key_iv: "ab7aed87e4a783fc4896a81ffdf7c609"
server_key_version: "encryptionkey@0"
I20260924 21:12:34.538082 11571 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:12:34.538950 11585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:34.539139 11571 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:12:34.539244 11571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "2381242e210942f3ac854ebd44212430"
format_stamp: "Formatted at 2026-09-24 21:12:34 on dist-test-slave-q276"
server_key: "8f286b3c92e973a76bf7574c04dcb66b"
server_key_iv: "ab7aed87e4a783fc4896a81ffdf7c609"
server_key_version: "encryptionkey@0"
I20260924 21:12:34.539318 11571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:34.542629 11571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:34.542901 11571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:34.543042 11571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:34.546941 11571 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:43743
I20260924 21:12:34.546994 11637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:43743 every 8 connection(s)
I20260924 21:12:34.547358 11571 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:12:34.547997 11638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:34.548655 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 11571
I20260924 21:12:34.548789 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:12:34.549048 6986 external_mini_cluster.cc:1428] Setting key a5024116b8c3598d41dd7d662ef69c41
I20260924 21:12:34.551674 11641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.549615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45284 (local address 127.6.210.190:43743)
0924 21:12:34.549813 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:34.549821 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:34.549845 (+ 24us) server_negotiation.cc:407] Connection header received
0924 21:12:34.550364 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:34.550371 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:34.550415 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:34.550531 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:34.551210 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:34.551214 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:34.551214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:12:34.551218 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:34.551258 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:12:34.551268 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:34.551323 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:34.551411 (+ 88us) server_negotiation.cc:299] Negotiation successful
0924 21:12:34.551456 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":82,"threads_started":1}
I20260924 21:12:34.552209 11638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2381242e210942f3ac854ebd44212430: Bootstrap starting.
I20260924 21:12:34.552820 11638 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2381242e210942f3ac854ebd44212430: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:34.553084 11638 log.cc:824] T 00000000000000000000000000000000 P 2381242e210942f3ac854ebd44212430: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:34.553684 11638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2381242e210942f3ac854ebd44212430: No bootstrap required, opened a new log
I20260924 21:12:34.555017 11638 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2381242e210942f3ac854ebd44212430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2381242e210942f3ac854ebd44212430" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 43743 } }
I20260924 21:12:34.555157 11638 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2381242e210942f3ac854ebd44212430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:34.555179 11638 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2381242e210942f3ac854ebd44212430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2381242e210942f3ac854ebd44212430, State: Initialized, Role: FOLLOWER
I20260924 21:12:34.555291 11638 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2381242e210942f3ac854ebd44212430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2381242e210942f3ac854ebd44212430" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 43743 } }
I20260924 21:12:34.555368 11638 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2381242e210942f3ac854ebd44212430 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:12:34.555389 11638 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2381242e210942f3ac854ebd44212430 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:12:34.555418 11638 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2381242e210942f3ac854ebd44212430 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:34.555898 11638 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2381242e210942f3ac854ebd44212430 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2381242e210942f3ac854ebd44212430" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 43743 } }
I20260924 21:12:34.555980 11638 leader_election.cc:302] T 00000000000000000000000000000000 P 2381242e210942f3ac854ebd44212430 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2381242e210942f3ac854ebd44212430; no voters:
I20260924 21:12:34.556133 11638 leader_election.cc:288] T 00000000000000000000000000000000 P 2381242e210942f3ac854ebd44212430 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:12:34.556409 11643 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2381242e210942f3ac854ebd44212430 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:34.556409 11638 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2381242e210942f3ac854ebd44212430 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:12:34.556555 11643 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2381242e210942f3ac854ebd44212430 [term 1 LEADER]: Becoming Leader. State: Replica: 2381242e210942f3ac854ebd44212430, State: Running, Role: LEADER
I20260924 21:12:34.556682 11643 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2381242e210942f3ac854ebd44212430 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2381242e210942f3ac854ebd44212430" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 43743 } }
I20260924 21:12:34.557029 11644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2381242e210942f3ac854ebd44212430 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2381242e210942f3ac854ebd44212430" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2381242e210942f3ac854ebd44212430" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 43743 } } }
I20260924 21:12:34.557147 11644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2381242e210942f3ac854ebd44212430 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:34.557029 11645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2381242e210942f3ac854ebd44212430 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2381242e210942f3ac854ebd44212430. Latest consensus state: current_term: 1 leader_uuid: "2381242e210942f3ac854ebd44212430" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2381242e210942f3ac854ebd44212430" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 43743 } } }
I20260924 21:12:34.557261 11645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2381242e210942f3ac854ebd44212430 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:34.557379 11652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:12:34.557876 11652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:12:34.559381 11652 catalog_manager.cc:1347] Generated new cluster ID: 35bf505b215f444e881224a0ead8a9dc
I20260924 21:12:34.559442 11652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:12:34.574847 11652 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:12:34.575133 11652 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:12:34.581367 11652 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2381242e210942f3ac854ebd44212430: Generated new TSK 0
I20260924 21:12:34.581589 11652 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:12:34.588308 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.129:0
--local_ip_for_outbound_sockets=127.6.210.129
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:43743
--builtin_ntp_servers=127.6.210.148:44879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260924 21:12:34.684089 11662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:34.684317 11662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:34.684355 11662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:34.685500 11662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:34.685562 11662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:34.685606 11662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.129
I20260924 21:12:34.687016 11662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:44879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:43743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.129
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:34.687268 11662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:34.687553 11662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:34.693425 11667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:34.693435 11668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:34.693691 11662 server_base.cc:1034] running on GCE node
W20260924 21:12:34.693467 11670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:34.694020 11662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:34.694350 11662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:34.695528 11662 hybrid_clock.cc:648] HybridClock initialized: now 1790284354695503 us; error 37 us; skew 500 ppm
I20260924 21:12:34.696910 11662 webserver.cc:460] Webserver started at http://127.6.210.129:44617/ using document root <none> and password file <none>
I20260924 21:12:34.697160 11662 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:34.697213 11662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:34.697333 11662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:34.698678 11662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-0/data/instance:
uuid: "51473251e01e404bb7915dc3760bf21a"
format_stamp: "Formatted at 2026-09-24 21:12:34 on dist-test-slave-q276"
server_key: "05b42cc79094ba5aff8cbc2421c34ac2"
server_key_iv: "4fb6548e0e20b21189454f11ba17c185"
server_key_version: "encryptionkey@0"
I20260924 21:12:34.699056 11662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance:
uuid: "51473251e01e404bb7915dc3760bf21a"
format_stamp: "Formatted at 2026-09-24 21:12:34 on dist-test-slave-q276"
server_key: "05b42cc79094ba5aff8cbc2421c34ac2"
server_key_iv: "4fb6548e0e20b21189454f11ba17c185"
server_key_version: "encryptionkey@0"
I20260924 21:12:34.700588 11662 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:12:34.701416 11676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:34.701622 11662 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:34.701709 11662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-0/wal
uuid: "51473251e01e404bb7915dc3760bf21a"
format_stamp: "Formatted at 2026-09-24 21:12:34 on dist-test-slave-q276"
server_key: "05b42cc79094ba5aff8cbc2421c34ac2"
server_key_iv: "4fb6548e0e20b21189454f11ba17c185"
server_key_version: "encryptionkey@0"
I20260924 21:12:34.701793 11662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:34.706377 11662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:34.706595 11662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:34.706693 11662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:34.706848 11662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:34.707154 11662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:34.707187 11662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:34.707207 11662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:34.707254 11662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:34.712702 11662 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.129:35193
I20260924 21:12:34.712769 11789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.129:35193 every 8 connection(s)
I20260924 21:12:34.713101 11662 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
I20260924 21:12:34.713483 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 11662
I20260924 21:12:34.713590 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance
I20260924 21:12:34.713750 6986 external_mini_cluster.cc:1428] Setting key 2f9e06edbabe9070d5a6960e0be960e8
I20260924 21:12:34.714691 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.130:0
--local_ip_for_outbound_sockets=127.6.210.130
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:43743
--builtin_ntp_servers=127.6.210.148:44879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260924 21:12:34.716279 11641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.713838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:42799 (local address 127.6.210.190:43743)
0924 21:12:34.713907 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:34.713910 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:34.714158 (+ 248us) server_negotiation.cc:407] Connection header received
0924 21:12:34.714777 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:34.714778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:34.714784 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:34.714830 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:34.716045 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:34.716046 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:34.716046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:12:34.716051 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:34.716067 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:12:34.716071 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:34.716118 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:34.716198 (+ 80us) server_negotiation.cc:299] Negotiation successful
0924 21:12:34.716212 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 21:12:34.716475 11792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.713824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:43743 (local address 127.6.210.129:42799)
0924 21:12:34.714016 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:34.714054 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:34.714672 (+ 618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:34.714896 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:34.714900 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:34.714909 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:34.715888 (+ 979us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:12:34.715888 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:34.715916 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:12:34.715942 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:34.716115 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:34.716120 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:34.716123 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:12:34.716201 (+ 78us) client_negotiation.cc:240] Negotiation successful
0924 21:12:34.716262 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":94,"threads_started":1}
I20260924 21:12:34.717073 11790 heartbeater.cc:347] Connected to a master server at 127.6.210.190:43743
I20260924 21:12:34.717178 11790 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:34.717324 11790 heartbeater.cc:511] Master 127.6.210.190:43743 requested a full tablet report, sending...
I20260924 21:12:34.717800 11602 ts_manager.cc:194] Registered new tserver with Master: 51473251e01e404bb7915dc3760bf21a (127.6.210.129:35193)
W20260924 21:12:34.794581 11793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:34.794785 11793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:34.794808 11793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:34.795996 11793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:34.796036 11793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:34.796093 11793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.130
I20260924 21:12:34.797436 11793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:44879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:43743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.130
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:34.797693 11793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:34.797986 11793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:34.802922 11798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:34.803061 11801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:34.802937 11799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:34.803172 11793 server_base.cc:1034] running on GCE node
I20260924 21:12:34.803359 11793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:34.803601 11793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:34.804760 11793 hybrid_clock.cc:648] HybridClock initialized: now 1790284354804743 us; error 34 us; skew 500 ppm
I20260924 21:12:34.806120 11793 webserver.cc:460] Webserver started at http://127.6.210.130:33469/ using document root <none> and password file <none>
I20260924 21:12:34.806386 11793 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:34.806444 11793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:34.806565 11793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:34.807662 11793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-1/data/instance:
uuid: "2c365203dc234c218883149ab1682527"
format_stamp: "Formatted at 2026-09-24 21:12:34 on dist-test-slave-q276"
server_key: "b8d7c50fa5cf306f443e6232dc3f1144"
server_key_iv: "7d035894554bf1f5c1c862eede2145cb"
server_key_version: "encryptionkey@0"
I20260924 21:12:34.807951 11793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance:
uuid: "2c365203dc234c218883149ab1682527"
format_stamp: "Formatted at 2026-09-24 21:12:34 on dist-test-slave-q276"
server_key: "b8d7c50fa5cf306f443e6232dc3f1144"
server_key_iv: "7d035894554bf1f5c1c862eede2145cb"
server_key_version: "encryptionkey@0"
I20260924 21:12:34.809190 11793 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:12:34.809875 11807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:34.810119 11793 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:34.810199 11793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-1/wal
uuid: "2c365203dc234c218883149ab1682527"
format_stamp: "Formatted at 2026-09-24 21:12:34 on dist-test-slave-q276"
server_key: "b8d7c50fa5cf306f443e6232dc3f1144"
server_key_iv: "7d035894554bf1f5c1c862eede2145cb"
server_key_version: "encryptionkey@0"
I20260924 21:12:34.810254 11793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:34.813587 11793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:34.813808 11793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:34.813900 11793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:34.814049 11793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:34.814344 11793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:34.814378 11793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:34.814399 11793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:34.814410 11793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:34.820127 11793 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.130:38349
I20260924 21:12:34.820187 11920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.130:38349 every 8 connection(s)
I20260924 21:12:34.820494 11793 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
I20260924 21:12:34.823545 11641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.821204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:43175 (local address 127.6.210.190:43743)
0924 21:12:34.821270 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:34.821272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:34.821493 (+ 221us) server_negotiation.cc:407] Connection header received
0924 21:12:34.821930 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:34.821931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:34.821935 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:34.822062 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:34.823339 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:34.823340 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:34.823341 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:12:34.823344 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:34.823359 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:12:34.823363 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:34.823410 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:34.823480 (+ 70us) server_negotiation.cc:299] Negotiation successful
0924 21:12:34.823493 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 21:12:34.823745 11923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.821180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:43743 (local address 127.6.210.130:43175)
0924 21:12:34.821354 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:34.821395 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:34.821847 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:34.822150 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:34.822154 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:34.822162 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:34.823196 (+ 1034us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:12:34.823197 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:34.823226 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:12:34.823254 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:34.823415 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:34.823418 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:34.823420 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 21:12:34.823469 (+ 49us) client_negotiation.cc:240] Negotiation successful
0924 21:12:34.823518 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":82,"threads_started":1}
I20260924 21:12:34.824153 11921 heartbeater.cc:347] Connected to a master server at 127.6.210.190:43743
I20260924 21:12:34.824236 11921 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:34.824340 11921 heartbeater.cc:511] Master 127.6.210.190:43743 requested a full tablet report, sending...
I20260924 21:12:34.824677 11602 ts_manager.cc:194] Registered new tserver with Master: 2c365203dc234c218883149ab1682527 (127.6.210.130:38349)
I20260924 21:12:34.829578 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 11793
I20260924 21:12:34.829694 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance
I20260924 21:12:34.829885 6986 external_mini_cluster.cc:1428] Setting key 92fdef258fe51a456e144818f6153b6e
I20260924 21:12:34.830780 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.131:0
--local_ip_for_outbound_sockets=127.6.210.131
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:43743
--builtin_ntp_servers=127.6.210.148:44879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260924 21:12:34.916090 11924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:34.916289 11924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:34.916311 11924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:34.917486 11924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:34.917526 11924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:34.917579 11924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.131
I20260924 21:12:34.918955 11924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:44879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:43743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.131
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:34.919201 11924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:34.919485 11924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:34.924435 11929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:34.924443 11932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:34.924443 11930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:34.924888 11924 server_base.cc:1034] running on GCE node
I20260924 21:12:34.925102 11924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:34.925297 11924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:34.926429 11924 hybrid_clock.cc:648] HybridClock initialized: now 1790284354926418 us; error 28 us; skew 500 ppm
I20260924 21:12:34.927657 11924 webserver.cc:460] Webserver started at http://127.6.210.131:46131/ using document root <none> and password file <none>
I20260924 21:12:34.927891 11924 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:34.927934 11924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:34.928076 11924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:34.929145 11924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-2/data/instance:
uuid: "c0feefc18c59464eada9983ad9ea98e8"
format_stamp: "Formatted at 2026-09-24 21:12:34 on dist-test-slave-q276"
server_key: "52e3a4d8ca13b236ee59a30d66feb087"
server_key_iv: "6cc3ab13d84ed5e85a84e08672f6f1d1"
server_key_version: "encryptionkey@0"
I20260924 21:12:34.929505 11924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance:
uuid: "c0feefc18c59464eada9983ad9ea98e8"
format_stamp: "Formatted at 2026-09-24 21:12:34 on dist-test-slave-q276"
server_key: "52e3a4d8ca13b236ee59a30d66feb087"
server_key_iv: "6cc3ab13d84ed5e85a84e08672f6f1d1"
server_key_version: "encryptionkey@0"
I20260924 21:12:34.930866 11924 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:12:34.931556 11938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:34.931754 11924 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:12:34.931859 11924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-2/wal
uuid: "c0feefc18c59464eada9983ad9ea98e8"
format_stamp: "Formatted at 2026-09-24 21:12:34 on dist-test-slave-q276"
server_key: "52e3a4d8ca13b236ee59a30d66feb087"
server_key_iv: "6cc3ab13d84ed5e85a84e08672f6f1d1"
server_key_version: "encryptionkey@0"
I20260924 21:12:34.931954 11924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:34.935276 11924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:34.935536 11924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:34.935683 11924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:34.935894 11924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:34.936184 11924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:34.936236 11924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:34.936275 11924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:34.936302 11924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:34.942150 11924 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.131:41681
I20260924 21:12:34.942191 12051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.131:41681 every 8 connection(s)
I20260924 21:12:34.942530 11924 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
I20260924 21:12:34.945003 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 11924
I20260924 21:12:34.945092 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance
I20260924 21:12:34.945261 6986 external_mini_cluster.cc:1428] Setting key 78c98ef2e039981cc47389274cd49aad
I20260924 21:12:34.945711 11641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.943233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:49743 (local address 127.6.210.190:43743)
0924 21:12:34.943309 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:34.943311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:34.943477 (+ 166us) server_negotiation.cc:407] Connection header received
0924 21:12:34.943971 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:34.943972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:34.943983 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:34.944048 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:34.945451 (+ 1403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:34.945453 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:34.945453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:12:34.945457 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:34.945470 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:12:34.945473 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:34.945546 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:34.945627 (+ 81us) server_negotiation.cc:299] Negotiation successful
0924 21:12:34.945642 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260924 21:12:34.945901 12054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.943191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:43743 (local address 127.6.210.131:49743)
0924 21:12:34.943360 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:34.943394 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:34.943880 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:34.944107 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:34.944111 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:34.944121 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:34.945268 (+ 1147us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:12:34.945268 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:34.945324 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:12:34.945362 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:34.945545 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:34.945552 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:34.945555 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:12:34.945628 (+ 73us) client_negotiation.cc:240] Negotiation successful
0924 21:12:34.945690 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":87,"threads_started":1}
I20260924 21:12:34.946378 12052 heartbeater.cc:347] Connected to a master server at 127.6.210.190:43743
I20260924 21:12:34.946486 12052 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:34.946640 12052 heartbeater.cc:511] Master 127.6.210.190:43743 requested a full tablet report, sending...
I20260924 21:12:34.946995 11602 ts_manager.cc:194] Registered new tserver with Master: c0feefc18c59464eada9983ad9ea98e8 (127.6.210.131:41681)
I20260924 21:12:34.957074 6986 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:12:34.959640 11641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.958468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45290 (local address 127.6.210.190:43743)
0924 21:12:34.958508 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:34.958523 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:34.958660 (+ 137us) server_negotiation.cc:407] Connection header received
0924 21:12:34.958741 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:34.958742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:34.958748 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:34.958818 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:34.959386 (+ 568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:34.959387 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:34.959388 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:12:34.959392 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:34.959405 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:12:34.959409 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:34.959455 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:34.959509 (+ 54us) server_negotiation.cc:299] Negotiation successful
0924 21:12:34.959525 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 21:12:34.961342 11602 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45290:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:12:34.963193 11602 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:12:34.967603 12063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.966072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:38349 (local address 127.0.0.1:59108)
0924 21:12:34.966416 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:34.966427 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:34.966464 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:34.966624 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:34.966629 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:34.966636 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:34.967156 (+ 520us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:12:34.967157 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:34.967173 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:12:34.967199 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:34.967424 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:34.967431 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:34.967434 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:12:34.967509 (+ 75us) client_negotiation.cc:240] Negotiation successful
0924 21:12:34.967523 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":73,"threads_started":1}
I20260924 21:12:34.967658 12064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.966182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59108 (local address 127.6.210.130:38349)
0924 21:12:34.966474 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:34.966479 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:34.966490 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:12:34.966518 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:34.966521 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:34.966531 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:34.966615 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:34.967271 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:34.967278 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:34.967278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:12:34.967284 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:34.967323 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:12:34.967358 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:34.967414 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:34.967544 (+ 130us) server_negotiation.cc:299] Negotiation successful
0924 21:12:34.967563 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":98,"threads_started":1}
I20260924 21:12:34.968266 12060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.965667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:41681 (local address 127.0.0.1:42272)
0924 21:12:34.965943 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:34.965988 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:34.966040 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:34.966974 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:34.966975 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:34.966977 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:34.967460 (+ 483us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:12:34.967461 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:34.967466 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:12:34.967477 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:34.968130 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:34.968131 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:34.968132 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 21:12:34.968182 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 21:12:34.968197 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":79,"threads_started":1}
I20260924 21:12:34.968542 12062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.966027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57980 (local address 127.6.210.129:35193)
0924 21:12:34.967178 (+ 1151us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:34.967183 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:34.967193 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:12:34.967218 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:34.967220 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:34.967229 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:34.967297 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:34.968062 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:34.968070 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:34.968071 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:12:34.968075 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:34.968107 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:12:34.968168 (+ 61us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:34.968225 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:34.968460 (+ 235us) server_negotiation.cc:299] Negotiation successful
0924 21:12:34.968475 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1107,"thread_start_us":104,"threads_started":1}
I20260924 21:12:34.968717 12065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.966212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42272 (local address 127.6.210.131:41681)
0924 21:12:34.966562 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:34.966566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:34.966574 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:12:34.966618 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:34.966620 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:34.966629 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:34.966707 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:34.967602 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:34.967609 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:34.967610 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:12:34.967614 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:34.967996 (+ 382us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:12:34.968033 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:34.968101 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:34.968483 (+ 382us) server_negotiation.cc:299] Negotiation successful
0924 21:12:34.968501 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":110,"threads_started":1}
I20260924 21:12:34.968673 11855 tablet_service.cc:1467] Processing CreateTablet for tablet 0c11a82f376b4f71a1ba6825487882be (DEFAULT_TABLE table=test-table [id=bc564e75a6244cb481f2a1f0307dded1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:34.968395 12061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.965902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:35193 (local address 127.0.0.1:57980)
0924 21:12:34.966231 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:34.966242 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:34.966311 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:34.967574 (+ 1263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:34.967574 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:34.967576 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:34.967969 (+ 393us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:12:34.967970 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:34.967975 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:12:34.967985 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:34.968252 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:34.968253 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:34.968254 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 21:12:34.968301 (+ 47us) client_negotiation.cc:240] Negotiation successful
0924 21:12:34.968313 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":85,"threads_started":1}
I20260924 21:12:34.969074 11855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c11a82f376b4f71a1ba6825487882be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:34.969367 11986 tablet_service.cc:1467] Processing CreateTablet for tablet 0c11a82f376b4f71a1ba6825487882be (DEFAULT_TABLE table=test-table [id=bc564e75a6244cb481f2a1f0307dded1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:34.969679 11986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c11a82f376b4f71a1ba6825487882be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:34.971951 12066 tablet_bootstrap.cc:492] T 0c11a82f376b4f71a1ba6825487882be P 2c365203dc234c218883149ab1682527: Bootstrap starting.
I20260924 21:12:34.972357 12067 tablet_bootstrap.cc:492] T 0c11a82f376b4f71a1ba6825487882be P c0feefc18c59464eada9983ad9ea98e8: Bootstrap starting.
I20260924 21:12:34.972679 12066 tablet_bootstrap.cc:654] T 0c11a82f376b4f71a1ba6825487882be P 2c365203dc234c218883149ab1682527: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:34.972795 11724 tablet_service.cc:1467] Processing CreateTablet for tablet 0c11a82f376b4f71a1ba6825487882be (DEFAULT_TABLE table=test-table [id=bc564e75a6244cb481f2a1f0307dded1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:34.973042 12067 tablet_bootstrap.cc:654] T 0c11a82f376b4f71a1ba6825487882be P c0feefc18c59464eada9983ad9ea98e8: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:34.973042 12066 log.cc:824] T 0c11a82f376b4f71a1ba6825487882be P 2c365203dc234c218883149ab1682527: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:34.973116 11724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c11a82f376b4f71a1ba6825487882be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:34.973371 12067 log.cc:824] T 0c11a82f376b4f71a1ba6825487882be P c0feefc18c59464eada9983ad9ea98e8: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:34.974002 12066 tablet_bootstrap.cc:492] T 0c11a82f376b4f71a1ba6825487882be P 2c365203dc234c218883149ab1682527: No bootstrap required, opened a new log
I20260924 21:12:34.974078 12066 ts_tablet_manager.cc:1397] T 0c11a82f376b4f71a1ba6825487882be P 2c365203dc234c218883149ab1682527: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:12:34.974112 12067 tablet_bootstrap.cc:492] T 0c11a82f376b4f71a1ba6825487882be P c0feefc18c59464eada9983ad9ea98e8: No bootstrap required, opened a new log
I20260924 21:12:34.974171 12067 ts_tablet_manager.cc:1397] T 0c11a82f376b4f71a1ba6825487882be P c0feefc18c59464eada9983ad9ea98e8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:12:34.975710 12070 tablet_bootstrap.cc:492] T 0c11a82f376b4f71a1ba6825487882be P 51473251e01e404bb7915dc3760bf21a: Bootstrap starting.
I20260924 21:12:34.976291 12070 tablet_bootstrap.cc:654] T 0c11a82f376b4f71a1ba6825487882be P 51473251e01e404bb7915dc3760bf21a: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:34.976580 12070 log.cc:824] T 0c11a82f376b4f71a1ba6825487882be P 51473251e01e404bb7915dc3760bf21a: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:34.977044 12067 raft_consensus.cc:348] T 0c11a82f376b4f71a1ba6825487882be P c0feefc18c59464eada9983ad9ea98e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0feefc18c59464eada9983ad9ea98e8" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 41681 } } peers { permanent_uuid: "51473251e01e404bb7915dc3760bf21a" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 35193 } } peers { permanent_uuid: "2c365203dc234c218883149ab1682527" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38349 } }
I20260924 21:12:34.977227 12067 raft_consensus.cc:374] T 0c11a82f376b4f71a1ba6825487882be P c0feefc18c59464eada9983ad9ea98e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:34.977265 12067 raft_consensus.cc:729] T 0c11a82f376b4f71a1ba6825487882be P c0feefc18c59464eada9983ad9ea98e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0feefc18c59464eada9983ad9ea98e8, State: Initialized, Role: FOLLOWER
I20260924 21:12:34.977324 12070 tablet_bootstrap.cc:492] T 0c11a82f376b4f71a1ba6825487882be P 51473251e01e404bb7915dc3760bf21a: No bootstrap required, opened a new log
I20260924 21:12:34.977386 12070 ts_tablet_manager.cc:1397] T 0c11a82f376b4f71a1ba6825487882be P 51473251e01e404bb7915dc3760bf21a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:12:34.977339 12066 raft_consensus.cc:348] T 0c11a82f376b4f71a1ba6825487882be P 2c365203dc234c218883149ab1682527 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0feefc18c59464eada9983ad9ea98e8" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 41681 } } peers { permanent_uuid: "51473251e01e404bb7915dc3760bf21a" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 35193 } } peers { permanent_uuid: "2c365203dc234c218883149ab1682527" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38349 } }
I20260924 21:12:34.977458 12067 consensus_queue.cc:260] T 0c11a82f376b4f71a1ba6825487882be P c0feefc18c59464eada9983ad9ea98e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0feefc18c59464eada9983ad9ea98e8" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 41681 } } peers { permanent_uuid: "51473251e01e404bb7915dc3760bf21a" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 35193 } } peers { permanent_uuid: "2c365203dc234c218883149ab1682527" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38349 } }
I20260924 21:12:34.977509 12066 raft_consensus.cc:374] T 0c11a82f376b4f71a1ba6825487882be P 2c365203dc234c218883149ab1682527 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:34.977552 12066 raft_consensus.cc:729] T 0c11a82f376b4f71a1ba6825487882be P 2c365203dc234c218883149ab1682527 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c365203dc234c218883149ab1682527, State: Initialized, Role: FOLLOWER
I20260924 21:12:34.977656 12072 raft_consensus.cc:482] T 0c11a82f376b4f71a1ba6825487882be P c0feefc18c59464eada9983ad9ea98e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:12:34.977723 12067 ts_tablet_manager.cc:1428] T 0c11a82f376b4f71a1ba6825487882be P c0feefc18c59464eada9983ad9ea98e8: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260924 21:12:34.977691 12066 consensus_queue.cc:260] T 0c11a82f376b4f71a1ba6825487882be P 2c365203dc234c218883149ab1682527 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0feefc18c59464eada9983ad9ea98e8" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 41681 } } peers { permanent_uuid: "51473251e01e404bb7915dc3760bf21a" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 35193 } } peers { permanent_uuid: "2c365203dc234c218883149ab1682527" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38349 } }
I20260924 21:12:34.977761 12072 raft_consensus.cc:504] T 0c11a82f376b4f71a1ba6825487882be P c0feefc18c59464eada9983ad9ea98e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0feefc18c59464eada9983ad9ea98e8" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 41681 } } peers { permanent_uuid: "51473251e01e404bb7915dc3760bf21a" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 35193 } } peers { permanent_uuid: "2c365203dc234c218883149ab1682527" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38349 } }
I20260924 21:12:34.977855 12052 heartbeater.cc:503] Master 127.6.210.190:43743 was elected leader, sending a full tablet report...
I20260924 21:12:34.977948 12066 ts_tablet_manager.cc:1428] T 0c11a82f376b4f71a1ba6825487882be P 2c365203dc234c218883149ab1682527: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260924 21:12:34.978051 11921 heartbeater.cc:503] Master 127.6.210.190:43743 was elected leader, sending a full tablet report...
I20260924 21:12:34.978324 12072 leader_election.cc:288] T 0c11a82f376b4f71a1ba6825487882be P c0feefc18c59464eada9983ad9ea98e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51473251e01e404bb7915dc3760bf21a (127.6.210.129:35193), 2c365203dc234c218883149ab1682527 (127.6.210.130:38349)
I20260924 21:12:34.979954 12054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.978520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:35193 (local address 127.6.210.131:47783)
0924 21:12:34.978783 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:34.978795 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:34.978840 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:34.979030 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:34.979031 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:34.979033 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:34.979588 (+ 555us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:12:34.979588 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:34.979596 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:12:34.979612 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:34.979811 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:34.979812 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:34.979813 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 21:12:34.979860 (+ 47us) client_negotiation.cc:240] Negotiation successful
0924 21:12:34.979874 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247}
I20260924 21:12:34.980427 12062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.978745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:47783 (local address 127.6.210.129:35193)
0924 21:12:34.978787 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:34.978790 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:34.978836 (+ 46us) server_negotiation.cc:407] Connection header received
0924 21:12:34.978907 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:34.978908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:34.978914 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:34.978970 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:34.979707 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:34.979709 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:34.979709 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:12:34.979714 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:34.979731 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:12:34.979736 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:34.979787 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:34.980318 (+ 531us) server_negotiation.cc:299] Negotiation successful
0924 21:12:34.980336 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 21:12:34.980752 12075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.978632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:38349 (local address 127.6.210.131:38251)
0924 21:12:34.979647 (+ 1015us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:34.979658 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:34.979713 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:34.979839 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:34.979841 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:34.979843 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:34.980325 (+ 482us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:12:34.980326 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:34.980332 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:12:34.980344 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:34.980522 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:34.980523 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:34.980524 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 21:12:34.980680 (+ 156us) client_negotiation.cc:240] Negotiation successful
0924 21:12:34.980692 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":998,"thread_start_us":110,"threads_started":1}
I20260924 21:12:34.980767 12064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.978967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:38251 (local address 127.6.210.130:38349)
0924 21:12:34.979030 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:34.979032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:34.979729 (+ 697us) server_negotiation.cc:407] Connection header received
0924 21:12:34.979762 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:34.979763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:34.979767 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:34.979822 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:34.980425 (+ 603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:34.980426 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:34.980427 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:12:34.980431 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:34.980445 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:12:34.980449 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:34.980514 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:34.980692 (+ 178us) server_negotiation.cc:299] Negotiation successful
0924 21:12:34.980707 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260924 21:12:34.980757 11744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c11a82f376b4f71a1ba6825487882be" candidate_uuid: "c0feefc18c59464eada9983ad9ea98e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51473251e01e404bb7915dc3760bf21a" is_pre_election: true
I20260924 21:12:34.981074 11875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c11a82f376b4f71a1ba6825487882be" candidate_uuid: "c0feefc18c59464eada9983ad9ea98e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c365203dc234c218883149ab1682527" is_pre_election: true
I20260924 21:12:34.981189 11875 raft_consensus.cc:2454] T 0c11a82f376b4f71a1ba6825487882be P 2c365203dc234c218883149ab1682527 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0feefc18c59464eada9983ad9ea98e8 in term 0.
I20260924 21:12:34.980603 12070 raft_consensus.cc:348] T 0c11a82f376b4f71a1ba6825487882be P 51473251e01e404bb7915dc3760bf21a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0feefc18c59464eada9983ad9ea98e8" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 41681 } } peers { permanent_uuid: "51473251e01e404bb7915dc3760bf21a" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 35193 } } peers { permanent_uuid: "2c365203dc234c218883149ab1682527" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38349 } }
I20260924 21:12:34.981247 12070 raft_consensus.cc:374] T 0c11a82f376b4f71a1ba6825487882be P 51473251e01e404bb7915dc3760bf21a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:34.981300 12070 raft_consensus.cc:729] T 0c11a82f376b4f71a1ba6825487882be P 51473251e01e404bb7915dc3760bf21a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51473251e01e404bb7915dc3760bf21a, State: Initialized, Role: FOLLOWER
I20260924 21:12:34.981387 11940 leader_election.cc:302] T 0c11a82f376b4f71a1ba6825487882be P c0feefc18c59464eada9983ad9ea98e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c365203dc234c218883149ab1682527, c0feefc18c59464eada9983ad9ea98e8; no voters:
I20260924 21:12:34.981416 12070 consensus_queue.cc:260] T 0c11a82f376b4f71a1ba6825487882be P 51473251e01e404bb7915dc3760bf21a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0feefc18c59464eada9983ad9ea98e8" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 41681 } } peers { permanent_uuid: "51473251e01e404bb7915dc3760bf21a" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 35193 } } peers { permanent_uuid: "2c365203dc234c218883149ab1682527" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38349 } }
I20260924 21:12:34.981528 12072 raft_consensus.cc:2781] T 0c11a82f376b4f71a1ba6825487882be P c0feefc18c59464eada9983ad9ea98e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:12:34.981556 11744 raft_consensus.cc:2454] T 0c11a82f376b4f71a1ba6825487882be P 51473251e01e404bb7915dc3760bf21a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0feefc18c59464eada9983ad9ea98e8 in term 0.
I20260924 21:12:34.981613 12072 raft_consensus.cc:482] T 0c11a82f376b4f71a1ba6825487882be P c0feefc18c59464eada9983ad9ea98e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:12:34.981679 12072 raft_consensus.cc:3037] T 0c11a82f376b4f71a1ba6825487882be P c0feefc18c59464eada9983ad9ea98e8 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:34.981770 12070 ts_tablet_manager.cc:1428] T 0c11a82f376b4f71a1ba6825487882be P 51473251e01e404bb7915dc3760bf21a: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260924 21:12:34.981779 11790 heartbeater.cc:503] Master 127.6.210.190:43743 was elected leader, sending a full tablet report...
I20260924 21:12:34.982486 12072 raft_consensus.cc:504] T 0c11a82f376b4f71a1ba6825487882be P c0feefc18c59464eada9983ad9ea98e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0feefc18c59464eada9983ad9ea98e8" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 41681 } } peers { permanent_uuid: "51473251e01e404bb7915dc3760bf21a" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 35193 } } peers { permanent_uuid: "2c365203dc234c218883149ab1682527" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38349 } }
I20260924 21:12:34.982625 12072 leader_election.cc:288] T 0c11a82f376b4f71a1ba6825487882be P c0feefc18c59464eada9983ad9ea98e8 [CANDIDATE]: Term 1 election: Requested vote from peers 51473251e01e404bb7915dc3760bf21a (127.6.210.129:35193), 2c365203dc234c218883149ab1682527 (127.6.210.130:38349)
I20260924 21:12:34.982784 11744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c11a82f376b4f71a1ba6825487882be" candidate_uuid: "c0feefc18c59464eada9983ad9ea98e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51473251e01e404bb7915dc3760bf21a"
I20260924 21:12:34.982856 11744 raft_consensus.cc:3037] T 0c11a82f376b4f71a1ba6825487882be P 51473251e01e404bb7915dc3760bf21a [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:34.982867 11875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c11a82f376b4f71a1ba6825487882be" candidate_uuid: "c0feefc18c59464eada9983ad9ea98e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c365203dc234c218883149ab1682527"
I20260924 21:12:34.982956 11875 raft_consensus.cc:3037] T 0c11a82f376b4f71a1ba6825487882be P 2c365203dc234c218883149ab1682527 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:34.983492 11744 raft_consensus.cc:2454] T 0c11a82f376b4f71a1ba6825487882be P 51473251e01e404bb7915dc3760bf21a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0feefc18c59464eada9983ad9ea98e8 in term 1.
I20260924 21:12:34.983641 11940 leader_election.cc:302] T 0c11a82f376b4f71a1ba6825487882be P c0feefc18c59464eada9983ad9ea98e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51473251e01e404bb7915dc3760bf21a, c0feefc18c59464eada9983ad9ea98e8; no voters:
I20260924 21:12:34.983716 12072 raft_consensus.cc:2781] T 0c11a82f376b4f71a1ba6825487882be P c0feefc18c59464eada9983ad9ea98e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:34.983721 11875 raft_consensus.cc:2454] T 0c11a82f376b4f71a1ba6825487882be P 2c365203dc234c218883149ab1682527 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0feefc18c59464eada9983ad9ea98e8 in term 1.
I20260924 21:12:34.983767 12072 raft_consensus.cc:686] T 0c11a82f376b4f71a1ba6825487882be P c0feefc18c59464eada9983ad9ea98e8 [term 1 LEADER]: Becoming Leader. State: Replica: c0feefc18c59464eada9983ad9ea98e8, State: Running, Role: LEADER
I20260924 21:12:34.983844 12072 consensus_queue.cc:237] T 0c11a82f376b4f71a1ba6825487882be P c0feefc18c59464eada9983ad9ea98e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0feefc18c59464eada9983ad9ea98e8" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 41681 } } peers { permanent_uuid: "51473251e01e404bb7915dc3760bf21a" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 35193 } } peers { permanent_uuid: "2c365203dc234c218883149ab1682527" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38349 } }
I20260924 21:12:34.984532 11601 catalog_manager.cc:5495] T 0c11a82f376b4f71a1ba6825487882be P c0feefc18c59464eada9983ad9ea98e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to c0feefc18c59464eada9983ad9ea98e8 (127.6.210.131). New cstate: current_term: 1 leader_uuid: "c0feefc18c59464eada9983ad9ea98e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0feefc18c59464eada9983ad9ea98e8" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 41681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "51473251e01e404bb7915dc3760bf21a" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 35193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c365203dc234c218883149ab1682527" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38349 } health_report { overall_health: UNKNOWN } } }
I20260924 21:12:34.988878 12065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:34.988071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42276 (local address 127.6.210.131:41681)
0924 21:12:34.988108 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:34.988110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:34.988119 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:12:34.988183 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:34.988184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:34.988192 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:34.988244 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:34.988695 (+ 451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:34.988696 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:34.988697 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:12:34.988700 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:34.988712 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:12:34.988716 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:34.988767 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:34.988800 (+ 33us) server_negotiation.cc:299] Negotiation successful
0924 21:12:34.988811 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 21:12:34.990415 11875 raft_consensus.cc:1261] T 0c11a82f376b4f71a1ba6825487882be P 2c365203dc234c218883149ab1682527 [term 1 FOLLOWER]: Refusing update from remote peer c0feefc18c59464eada9983ad9ea98e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:34.990437 11744 raft_consensus.cc:1261] T 0c11a82f376b4f71a1ba6825487882be P 51473251e01e404bb7915dc3760bf21a [term 1 FOLLOWER]: Refusing update from remote peer c0feefc18c59464eada9983ad9ea98e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:34.990764 12073 consensus_queue.cc:1035] T 0c11a82f376b4f71a1ba6825487882be P c0feefc18c59464eada9983ad9ea98e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c365203dc234c218883149ab1682527" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:12:34.990872 12072 consensus_queue.cc:1035] T 0c11a82f376b4f71a1ba6825487882be P c0feefc18c59464eada9983ad9ea98e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51473251e01e404bb7915dc3760bf21a" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 35193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:12:34.993543 12081 mvcc.cc:205] Tried to move back new op lower bound from 7333004718038552576 to 7333004718014574592. Current Snapshot: MvccSnapshot[applied={T|T < 7333004718038552576}]
I20260924 21:12:34.993638 12080 mvcc.cc:205] Tried to move back new op lower bound from 7333004718038552576 to 7333004718014574592. Current Snapshot: MvccSnapshot[applied={T|T < 7333004718038552576}]
I20260924 21:12:34.995476 11601 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:12:34.995571 11601 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:12:34.996286 11601 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2381242e210942f3ac854ebd44212430: Sending DeleteTablet for 3 replicas of tablet 0c11a82f376b4f71a1ba6825487882be
I20260924 21:12:34.996636 11724 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c11a82f376b4f71a1ba6825487882be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:34 UTC) from {username='slave'} at 127.0.0.1:57980
I20260924 21:12:34.996690 11855 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c11a82f376b4f71a1ba6825487882be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:34 UTC) from {username='slave'} at 127.0.0.1:59108
I20260924 21:12:34.996686 11986 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c11a82f376b4f71a1ba6825487882be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:34 UTC) from {username='slave'} at 127.0.0.1:42272
I20260924 21:12:34.996876 12090 tablet_replica.cc:330] T 0c11a82f376b4f71a1ba6825487882be P 51473251e01e404bb7915dc3760bf21a: stopping tablet replica
I20260924 21:12:34.996965 12091 tablet_replica.cc:330] T 0c11a82f376b4f71a1ba6825487882be P c0feefc18c59464eada9983ad9ea98e8: stopping tablet replica
I20260924 21:12:34.996974 12090 raft_consensus.cc:2229] T 0c11a82f376b4f71a1ba6825487882be P 51473251e01e404bb7915dc3760bf21a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:12:34.997072 12090 raft_consensus.cc:2258] T 0c11a82f376b4f71a1ba6825487882be P 51473251e01e404bb7915dc3760bf21a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:34.997119 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 11662
I20260924 21:12:34.997090 12091 raft_consensus.cc:2229] T 0c11a82f376b4f71a1ba6825487882be P c0feefc18c59464eada9983ad9ea98e8 [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:12:34.997335 12091 raft_consensus.cc:2258] T 0c11a82f376b4f71a1ba6825487882be P c0feefc18c59464eada9983ad9ea98e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:34.997705 12089 tablet_replica.cc:330] T 0c11a82f376b4f71a1ba6825487882be P 2c365203dc234c218883149ab1682527: stopping tablet replica
I20260924 21:12:34.997805 12089 raft_consensus.cc:2229] T 0c11a82f376b4f71a1ba6825487882be P 2c365203dc234c218883149ab1682527 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:12:34.997838 12091 ts_tablet_manager.cc:1905] T 0c11a82f376b4f71a1ba6825487882be P c0feefc18c59464eada9983ad9ea98e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:34.997903 12089 raft_consensus.cc:2258] T 0c11a82f376b4f71a1ba6825487882be P 2c365203dc234c218883149ab1682527 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:34.998342 12089 ts_tablet_manager.cc:1905] T 0c11a82f376b4f71a1ba6825487882be P 2c365203dc234c218883149ab1682527: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:34.999821 12091 ts_tablet_manager.cc:1918] T 0c11a82f376b4f71a1ba6825487882be P c0feefc18c59464eada9983ad9ea98e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:12:34.999938 12091 log.cc:1196] T 0c11a82f376b4f71a1ba6825487882be P c0feefc18c59464eada9983ad9ea98e8: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-2/wal/wals/0c11a82f376b4f71a1ba6825487882be
I20260924 21:12:35.000223 12089 ts_tablet_manager.cc:1918] T 0c11a82f376b4f71a1ba6825487882be P 2c365203dc234c218883149ab1682527: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:12:35.000301 12089 log.cc:1196] T 0c11a82f376b4f71a1ba6825487882be P 2c365203dc234c218883149ab1682527: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284333163253-6986-0/minicluster-data/ts-1/wal/wals/0c11a82f376b4f71a1ba6825487882be
I20260924 21:12:35.000392 12091 ts_tablet_manager.cc:1939] T 0c11a82f376b4f71a1ba6825487882be P c0feefc18c59464eada9983ad9ea98e8: Deleting consensus metadata
I20260924 21:12:35.000582 12089 ts_tablet_manager.cc:1939] T 0c11a82f376b4f71a1ba6825487882be P 2c365203dc234c218883149ab1682527: Deleting consensus metadata
I20260924 21:12:35.000885 11587 catalog_manager.cc:4841] TS c0feefc18c59464eada9983ad9ea98e8 (127.6.210.131:41681): tablet 0c11a82f376b4f71a1ba6825487882be (table test-table [id=bc564e75a6244cb481f2a1f0307dded1]) successfully deleted
I20260924 21:12:35.000978 11587 catalog_manager.cc:4841] TS 2c365203dc234c218883149ab1682527 (127.6.210.130:38349): tablet 0c11a82f376b4f71a1ba6825487882be (table test-table [id=bc564e75a6244cb481f2a1f0307dded1]) successfully deleted
W20260924 21:12:35.004109 11587 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.210.129:35193 (error 108)
I20260924 21:12:35.004673 12061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:35.004517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:35193 (local address 127.0.0.1:57986)
0924 21:12:35.004595 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:35.004640 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.129:35193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260924 21:12:35.004688 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 11793
W20260924 21:12:35.004817 11587 catalog_manager.cc:4584] TS 51473251e01e404bb7915dc3760bf21a (127.6.210.129:35193): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0c11a82f376b4f71a1ba6825487882be: Network error: Client connection negotiation failed: client connection to 127.6.210.129:35193: connect: Connection refused (error 111)
I20260924 21:12:35.010705 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 11924
I20260924 21:12:35.015679 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 11571
2026-09-24T21:12:35Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (597 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260924 21:12:35.035384 6986 mini_oidc.cc:152] Starting JWKS server
I20260924 21:12:35.036471 6986 webserver.cc:460] Webserver started at https://127.0.0.1:46009/ using document root <none> and password file <none>
I20260924 21:12:35.036586 6986 mini_oidc.cc:189] Starting OIDC Discovery server
I20260924 21:12:35.036834 6986 webserver.cc:460] Webserver started at http://127.0.0.1:42571/ using document root <none> and password file <none>
2026-09-24T21:12:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:12:35Z Disabled control of system clock
I20260924 21:12:35.044579 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:33321
--webserver_interface=127.6.210.190
--webserver_port=0
--builtin_ntp_servers=127.6.210.148:35749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:33321
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46009/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284333163253-6986-0/testchainca.cert with env {}
W20260924 21:12:35.124186 12098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:35.124383 12098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:35.124405 12098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:35.125476 12098 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:12:35.125509 12098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:35.125522 12098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:35.125535 12098 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:12:35.125546 12098 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260924 21:12:35.125558 12098 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260924 21:12:35.125569 12098 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46009/jwks.json
I20260924 21:12:35.127044 12098 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:35749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:33321
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:33321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46009/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284333163253-6986-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:35.127234 12098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:35.127388 12098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:35.132252 12103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:35.132252 12106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:35.132340 12098 server_base.cc:1034] running on GCE node
W20260924 21:12:35.132272 12104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:35.132740 12098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:35.133044 12098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:35.134235 12098 hybrid_clock.cc:648] HybridClock initialized: now 1790284355134221 us; error 32 us; skew 500 ppm
I20260924 21:12:35.135456 12098 webserver.cc:460] Webserver started at http://127.6.210.190:33849/ using document root <none> and password file <none>
I20260924 21:12:35.135700 12098 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:35.135746 12098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:35.135885 12098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:35.136938 12098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "c7ca0deaf55c40ffa196e6db274f0f81"
format_stamp: "Formatted at 2026-09-24 21:12:35 on dist-test-slave-q276"
server_key: "e7a379ca253315539c67794bc67ff66e"
server_key_iv: "47cce81541eba6743f2f39d9e1a726df"
server_key_version: "encryptionkey@0"
I20260924 21:12:35.137295 12098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "c7ca0deaf55c40ffa196e6db274f0f81"
format_stamp: "Formatted at 2026-09-24 21:12:35 on dist-test-slave-q276"
server_key: "e7a379ca253315539c67794bc67ff66e"
server_key_iv: "47cce81541eba6743f2f39d9e1a726df"
server_key_version: "encryptionkey@0"
I20260924 21:12:35.138612 12098 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:12:35.139295 12112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:35.139465 12098 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:35.139549 12098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "c7ca0deaf55c40ffa196e6db274f0f81"
format_stamp: "Formatted at 2026-09-24 21:12:35 on dist-test-slave-q276"
server_key: "e7a379ca253315539c67794bc67ff66e"
server_key_iv: "47cce81541eba6743f2f39d9e1a726df"
server_key_version: "encryptionkey@0"
I20260924 21:12:35.139604 12098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:35.147588 12098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:35.147825 12098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:35.147921 12098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:35.151536 12098 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:33321
I20260924 21:12:35.151605 12164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:33321 every 8 connection(s)
I20260924 21:12:35.151902 12098 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:12:35.152812 12165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:35.155054 12165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7ca0deaf55c40ffa196e6db274f0f81: Bootstrap starting.
I20260924 21:12:35.155611 12165 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7ca0deaf55c40ffa196e6db274f0f81: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:35.155894 12165 log.cc:824] T 00000000000000000000000000000000 P c7ca0deaf55c40ffa196e6db274f0f81: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:35.156522 12165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7ca0deaf55c40ffa196e6db274f0f81: No bootstrap required, opened a new log
I20260924 21:12:35.158049 12165 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7ca0deaf55c40ffa196e6db274f0f81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7ca0deaf55c40ffa196e6db274f0f81" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 33321 } }
I20260924 21:12:35.158205 12165 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7ca0deaf55c40ffa196e6db274f0f81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:35.158250 12165 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7ca0deaf55c40ffa196e6db274f0f81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7ca0deaf55c40ffa196e6db274f0f81, State: Initialized, Role: FOLLOWER
I20260924 21:12:35.158418 12165 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7ca0deaf55c40ffa196e6db274f0f81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7ca0deaf55c40ffa196e6db274f0f81" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 33321 } }
I20260924 21:12:35.158499 12165 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7ca0deaf55c40ffa196e6db274f0f81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:12:35.158543 12165 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7ca0deaf55c40ffa196e6db274f0f81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:12:35.158601 12165 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7ca0deaf55c40ffa196e6db274f0f81 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:35.158782 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 12098
I20260924 21:12:35.158885 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:12:35.159049 6986 external_mini_cluster.cc:1428] Setting key cd8953e00f193f79b64d5361ec55dc44
I20260924 21:12:35.159293 12165 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7ca0deaf55c40ffa196e6db274f0f81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7ca0deaf55c40ffa196e6db274f0f81" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 33321 } }
I20260924 21:12:35.159411 12165 leader_election.cc:302] T 00000000000000000000000000000000 P c7ca0deaf55c40ffa196e6db274f0f81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7ca0deaf55c40ffa196e6db274f0f81; no voters:
I20260924 21:12:35.159821 12165 leader_election.cc:288] T 00000000000000000000000000000000 P c7ca0deaf55c40ffa196e6db274f0f81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:12:35.159861 12170 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7ca0deaf55c40ffa196e6db274f0f81 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:35.160006 12170 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7ca0deaf55c40ffa196e6db274f0f81 [term 1 LEADER]: Becoming Leader. State: Replica: c7ca0deaf55c40ffa196e6db274f0f81, State: Running, Role: LEADER
I20260924 21:12:35.160076 12165 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7ca0deaf55c40ffa196e6db274f0f81 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:12:35.160091 12170 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7ca0deaf55c40ffa196e6db274f0f81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7ca0deaf55c40ffa196e6db274f0f81" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 33321 } }
I20260924 21:12:35.160403 12170 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7ca0deaf55c40ffa196e6db274f0f81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7ca0deaf55c40ffa196e6db274f0f81. Latest consensus state: current_term: 1 leader_uuid: "c7ca0deaf55c40ffa196e6db274f0f81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7ca0deaf55c40ffa196e6db274f0f81" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 33321 } } }
I20260924 21:12:35.160485 12170 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7ca0deaf55c40ffa196e6db274f0f81 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:35.160594 12171 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7ca0deaf55c40ffa196e6db274f0f81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7ca0deaf55c40ffa196e6db274f0f81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7ca0deaf55c40ffa196e6db274f0f81" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 33321 } } }
I20260924 21:12:35.160683 12171 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7ca0deaf55c40ffa196e6db274f0f81 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:35.161361 12176 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:12:35.161995 12176 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:12:35.163612 12176 catalog_manager.cc:1347] Generated new cluster ID: 6001836264f84bdbb4f77745c52fa0bf
I20260924 21:12:35.163704 12176 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:12:35.164920 12169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:35.159514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52842 (local address 127.6.210.190:33321)
0924 21:12:35.159728 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:35.159737 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:35.159772 (+ 35us) server_negotiation.cc:407] Connection header received
0924 21:12:35.159896 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:35.159904 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:35.159978 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:35.160383 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:35.160473 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:35.161927 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:35.162950 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:35.163473 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:35.163626 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:35.164312 (+ 686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:35.164315 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:35.164316 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:12:35.164490 (+ 174us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:35.164522 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:12:35.164533 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:35.164603 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:35.164678 (+ 75us) server_negotiation.cc:299] Negotiation successful
0924 21:12:35.164755 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":109,"threads_started":1}
I20260924 21:12:35.165716 6986 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:12:35.189153 12176 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:12:35.189864 12176 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:12:35.192198 12176 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7ca0deaf55c40ffa196e6db274f0f81: Generated new TSK 0
I20260924 21:12:35.192370 12176 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260924 21:12:35.215363 12195 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260924 21:12:35.215466 12195 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260924 21:12:35.215708 12169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:35.200295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52846 (local address 127.6.210.190:33321)
0924 21:12:35.200340 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:35.200342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:35.200474 (+ 132us) server_negotiation.cc:407] Connection header received
0924 21:12:35.200535 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:35.200536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:35.200548 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:35.200627 (+ 79us) server_negotiation.cc:226] Negotiated authn=JWT
0924 21:12:35.200761 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:35.201541 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:35.202374 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:35.202677 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:35.202743 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:35.202918 (+ 175us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0924 21:12:35.215497 (+ 12579us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46009/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0924 21:12:35.215636 (+ 139us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52846: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46009/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":24}
W20260924 21:12:35.215763 12169 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52846: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46009/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260924 21:12:35.216081 12194 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:35.200225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:33321 (local address 127.0.0.1:52846)
0924 21:12:35.200401 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:35.200412 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:35.200467 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:35.200637 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:35.200638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:35.200640 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0924 21:12:35.200695 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:35.200697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:35.201641 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:35.201643 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:35.202233 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:35.202235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:35.202756 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:35.202757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:35.202802 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:35.202820 (+ 18us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0924 21:12:35.215657 (+ 12837us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46009/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0924 21:12:35.215745 (+ 88us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.210.190:33321: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46009/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":93,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.6.210.190:33321: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46009/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260924 21:12:35.216522 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 12098
2026-09-24T21:12:35Z chronyd exiting
I20260924 21:12:35.252310 6986 test_util.cc:175] -----------------------------------------------
I20260924 21:12:35.252389 6986 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284333163253-6986-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (218 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260924 21:12:35.253122 6986 mini_oidc.cc:152] Starting JWKS server
I20260924 21:12:35.253865 6986 webserver.cc:460] Webserver started at https://127.0.0.1:33265/ using document root <none> and password file <none>
I20260924 21:12:35.253907 6986 mini_oidc.cc:189] Starting OIDC Discovery server
I20260924 21:12:35.254091 6986 webserver.cc:460] Webserver started at http://127.0.0.1:38385/ using document root <none> and password file <none>
2026-09-24T21:12:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:12:35Z Disabled control of system clock
I20260924 21:12:35.262310 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:42667
--webserver_interface=127.6.210.190
--webserver_port=0
--builtin_ntp_servers=127.6.210.148:43467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:42667
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33265/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284333163253-6986-0/testchainca.cert with env {}
W20260924 21:12:35.344362 12201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:35.344619 12201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:35.344655 12201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:35.345815 12201 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:12:35.345884 12201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:35.345907 12201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:35.345928 12201 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:12:35.345959 12201 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260924 21:12:35.345983 12201 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260924 21:12:35.346009 12201 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33265/jwks.json
I20260924 21:12:35.347518 12201 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:43467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:42667
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:42667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33265/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284333163253-6986-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:35.347801 12201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:35.348018 12201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:35.352888 12206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:35.352934 12209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:35.352934 12207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:35.353063 12201 server_base.cc:1034] running on GCE node
I20260924 21:12:35.353289 12201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:35.353632 12201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:35.354799 12201 hybrid_clock.cc:648] HybridClock initialized: now 1790284355354782 us; error 27 us; skew 500 ppm
I20260924 21:12:35.355973 12201 webserver.cc:460] Webserver started at http://127.6.210.190:42967/ using document root <none> and password file <none>
I20260924 21:12:35.356232 12201 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:35.356307 12201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:35.356462 12201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:35.357497 12201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "df9b2fb230a44c898c6e267eb735a5e8"
format_stamp: "Formatted at 2026-09-24 21:12:35 on dist-test-slave-q276"
server_key: "df890d7c1e09fc729d86f1045b364055"
server_key_iv: "e64fe60db56eab1338c655c9226f8e4a"
server_key_version: "encryptionkey@0"
I20260924 21:12:35.357862 12201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "df9b2fb230a44c898c6e267eb735a5e8"
format_stamp: "Formatted at 2026-09-24 21:12:35 on dist-test-slave-q276"
server_key: "df890d7c1e09fc729d86f1045b364055"
server_key_iv: "e64fe60db56eab1338c655c9226f8e4a"
server_key_version: "encryptionkey@0"
I20260924 21:12:35.359221 12201 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:12:35.359951 12215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:35.360112 12201 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:12:35.360210 12201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "df9b2fb230a44c898c6e267eb735a5e8"
format_stamp: "Formatted at 2026-09-24 21:12:35 on dist-test-slave-q276"
server_key: "df890d7c1e09fc729d86f1045b364055"
server_key_iv: "e64fe60db56eab1338c655c9226f8e4a"
server_key_version: "encryptionkey@0"
I20260924 21:12:35.360292 12201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:35.368892 12201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:35.369194 12201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:35.369344 12201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:35.373437 12201 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:42667
I20260924 21:12:35.373464 12267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:42667 every 8 connection(s)
I20260924 21:12:35.373818 12201 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:12:35.374845 12268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:35.376374 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 12201
I20260924 21:12:35.376499 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:12:35.376688 6986 external_mini_cluster.cc:1428] Setting key f5a327563423d658b7acdb2e711c6a7f
I20260924 21:12:35.377281 12268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df9b2fb230a44c898c6e267eb735a5e8: Bootstrap starting.
I20260924 21:12:35.377859 12268 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df9b2fb230a44c898c6e267eb735a5e8: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:35.378218 12268 log.cc:824] T 00000000000000000000000000000000 P df9b2fb230a44c898c6e267eb735a5e8: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:35.378999 12268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df9b2fb230a44c898c6e267eb735a5e8: No bootstrap required, opened a new log
I20260924 21:12:35.381124 12268 raft_consensus.cc:348] T 00000000000000000000000000000000 P df9b2fb230a44c898c6e267eb735a5e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df9b2fb230a44c898c6e267eb735a5e8" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 42667 } }
I20260924 21:12:35.381258 12268 raft_consensus.cc:374] T 00000000000000000000000000000000 P df9b2fb230a44c898c6e267eb735a5e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:35.381279 12268 raft_consensus.cc:729] T 00000000000000000000000000000000 P df9b2fb230a44c898c6e267eb735a5e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df9b2fb230a44c898c6e267eb735a5e8, State: Initialized, Role: FOLLOWER
I20260924 21:12:35.381407 12268 consensus_queue.cc:260] T 00000000000000000000000000000000 P df9b2fb230a44c898c6e267eb735a5e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df9b2fb230a44c898c6e267eb735a5e8" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 42667 } }
I20260924 21:12:35.381502 12268 raft_consensus.cc:388] T 00000000000000000000000000000000 P df9b2fb230a44c898c6e267eb735a5e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:12:35.381534 12268 raft_consensus.cc:482] T 00000000000000000000000000000000 P df9b2fb230a44c898c6e267eb735a5e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:12:35.381592 12268 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df9b2fb230a44c898c6e267eb735a5e8 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:35.382334 12268 raft_consensus.cc:504] T 00000000000000000000000000000000 P df9b2fb230a44c898c6e267eb735a5e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df9b2fb230a44c898c6e267eb735a5e8" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 42667 } }
I20260924 21:12:35.382419 12268 leader_election.cc:302] T 00000000000000000000000000000000 P df9b2fb230a44c898c6e267eb735a5e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df9b2fb230a44c898c6e267eb735a5e8; no voters:
I20260924 21:12:35.382601 12268 leader_election.cc:288] T 00000000000000000000000000000000 P df9b2fb230a44c898c6e267eb735a5e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:12:35.382650 12273 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df9b2fb230a44c898c6e267eb735a5e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:35.382808 12273 raft_consensus.cc:686] T 00000000000000000000000000000000 P df9b2fb230a44c898c6e267eb735a5e8 [term 1 LEADER]: Becoming Leader. State: Replica: df9b2fb230a44c898c6e267eb735a5e8, State: Running, Role: LEADER
I20260924 21:12:35.382858 12268 sys_catalog.cc:564] T 00000000000000000000000000000000 P df9b2fb230a44c898c6e267eb735a5e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:12:35.382912 12273 consensus_queue.cc:237] T 00000000000000000000000000000000 P df9b2fb230a44c898c6e267eb735a5e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df9b2fb230a44c898c6e267eb735a5e8" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 42667 } }
I20260924 21:12:35.383215 12273 sys_catalog.cc:455] T 00000000000000000000000000000000 P df9b2fb230a44c898c6e267eb735a5e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df9b2fb230a44c898c6e267eb735a5e8. Latest consensus state: current_term: 1 leader_uuid: "df9b2fb230a44c898c6e267eb735a5e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df9b2fb230a44c898c6e267eb735a5e8" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 42667 } } }
I20260924 21:12:35.383272 12271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:35.377212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37110 (local address 127.6.210.190:42667)
0924 21:12:35.377404 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:35.377413 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:35.377446 (+ 33us) server_negotiation.cc:407] Connection header received
0924 21:12:35.377853 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:35.377859 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:35.377915 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:35.378070 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:35.378402 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:35.379649 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:35.380796 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:35.381412 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:35.381577 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:35.382320 (+ 743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:35.382324 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:35.382325 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:12:35.382552 (+ 227us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:35.382598 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:12:35.382615 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:35.382682 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:35.382945 (+ 263us) server_negotiation.cc:299] Negotiation successful
0924 21:12:35.383035 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":84,"threads_started":1}
I20260924 21:12:35.383224 12274 sys_catalog.cc:455] T 00000000000000000000000000000000 P df9b2fb230a44c898c6e267eb735a5e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df9b2fb230a44c898c6e267eb735a5e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df9b2fb230a44c898c6e267eb735a5e8" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 42667 } } }
I20260924 21:12:35.383299 12273 sys_catalog.cc:458] T 00000000000000000000000000000000 P df9b2fb230a44c898c6e267eb735a5e8 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:35.383348 12274 sys_catalog.cc:458] T 00000000000000000000000000000000 P df9b2fb230a44c898c6e267eb735a5e8 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:35.383561 12279 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:12:35.384027 12279 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:12:35.384095 6986 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:12:35.385751 12279 catalog_manager.cc:1347] Generated new cluster ID: d4c39496c5fc4618a3bffe6137d0abd8
I20260924 21:12:35.385803 12279 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:12:35.398561 12279 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:12:35.399343 12279 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:12:35.406289 12279 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df9b2fb230a44c898c6e267eb735a5e8: Generated new TSK 0
I20260924 21:12:35.406504 12279 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260924 21:12:35.438108 12298 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260924 21:12:35.438235 12298 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260924 21:12:35.438766 12271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:35.415222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37124 (local address 127.6.210.190:42667)
0924 21:12:35.415284 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:35.415287 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:35.415424 (+ 137us) server_negotiation.cc:407] Connection header received
0924 21:12:35.415468 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:35.415468 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:35.415480 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:35.415551 (+ 71us) server_negotiation.cc:226] Negotiated authn=JWT
0924 21:12:35.415700 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:35.416215 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:35.417380 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:35.417582 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:35.417639 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:35.417839 (+ 200us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0924 21:12:35.438475 (+ 20636us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33265/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0924 21:12:35.438669 (+ 194us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37124: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33265/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":42}
W20260924 21:12:35.438930 12271 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37124: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33265/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260924 21:12:35.439005 12297 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:35.415118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:42667 (local address 127.0.0.1:37124)
0924 21:12:35.415291 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:35.415303 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:35.415371 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:35.415566 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:35.415566 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:35.415568 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0924 21:12:35.415631 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:35.415633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:35.416299 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:35.416300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:35.417287 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:35.417290 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:35.417659 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:35.417660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:35.417720 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:35.417726 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0924 21:12:35.438651 (+ 20925us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33265/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0924 21:12:35.438843 (+ 192us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.210.190:42667: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33265/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":110,"threads_started":1}
I20260924 21:12:35.439489 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 12201
2026-09-24T21:12:35Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (224 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260924 21:12:35.477839 6986 mini_oidc.cc:152] Starting JWKS server
I20260924 21:12:35.478645 6986 webserver.cc:460] Webserver started at https://127.0.0.1:45081/ using document root <none> and password file <none>
I20260924 21:12:35.478721 6986 mini_oidc.cc:189] Starting OIDC Discovery server
I20260924 21:12:35.478868 6986 webserver.cc:460] Webserver started at http://127.0.0.1:46267/ using document root <none> and password file <none>
2026-09-24T21:12:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:12:35Z Disabled control of system clock
I20260924 21:12:35.486514 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:39473
--webserver_interface=127.6.210.190
--webserver_port=0
--builtin_ntp_servers=127.6.210.148:37121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:39473
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45081/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260924 21:12:35.566512 12304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:35.566713 12304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:35.566735 12304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:35.567840 12304 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:12:35.567874 12304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:35.567886 12304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:35.567898 12304 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:12:35.567910 12304 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260924 21:12:35.567929 12304 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260924 21:12:35.567942 12304 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45081/jwks.json
I20260924 21:12:35.569199 12304 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:37121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:39473
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:39473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45081/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:35.569380 12304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:35.569604 12304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:35.574411 12310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:35.574412 12312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:35.574414 12309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:35.574589 12304 server_base.cc:1034] running on GCE node
I20260924 21:12:35.574872 12304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:35.575212 12304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:35.576382 12304 hybrid_clock.cc:648] HybridClock initialized: now 1790284355576360 us; error 41 us; skew 500 ppm
I20260924 21:12:35.577642 12304 webserver.cc:460] Webserver started at http://127.6.210.190:33639/ using document root <none> and password file <none>
I20260924 21:12:35.577893 12304 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:35.577965 12304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:35.578140 12304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:35.579326 12304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "bdb7a424412f48338c916abb30057385"
format_stamp: "Formatted at 2026-09-24 21:12:35 on dist-test-slave-q276"
server_key: "522da0a0e9cc463f25128601bfee625e"
server_key_iv: "700dca6c1dcd66407ea4a50b58e37fd6"
server_key_version: "encryptionkey@0"
I20260924 21:12:35.579699 12304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "bdb7a424412f48338c916abb30057385"
format_stamp: "Formatted at 2026-09-24 21:12:35 on dist-test-slave-q276"
server_key: "522da0a0e9cc463f25128601bfee625e"
server_key_iv: "700dca6c1dcd66407ea4a50b58e37fd6"
server_key_version: "encryptionkey@0"
I20260924 21:12:35.581012 12304 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:12:35.581739 12318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:35.581908 12304 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:12:35.582007 12304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "bdb7a424412f48338c916abb30057385"
format_stamp: "Formatted at 2026-09-24 21:12:35 on dist-test-slave-q276"
server_key: "522da0a0e9cc463f25128601bfee625e"
server_key_iv: "700dca6c1dcd66407ea4a50b58e37fd6"
server_key_version: "encryptionkey@0"
I20260924 21:12:35.582093 12304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:35.605732 12304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:35.606076 12304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:35.606319 12304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:35.610170 12304 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:39473
I20260924 21:12:35.610198 12370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:39473 every 8 connection(s)
I20260924 21:12:35.610612 12304 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:12:35.610754 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 12304
I20260924 21:12:35.610839 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:12:35.611068 6986 external_mini_cluster.cc:1428] Setting key 78078a8ac3e66c150f38ac2b95c44874
I20260924 21:12:35.611383 12371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:35.614250 12371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdb7a424412f48338c916abb30057385: Bootstrap starting.
I20260924 21:12:35.614964 12371 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bdb7a424412f48338c916abb30057385: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:35.615271 12371 log.cc:824] T 00000000000000000000000000000000 P bdb7a424412f48338c916abb30057385: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:35.616068 12371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdb7a424412f48338c916abb30057385: No bootstrap required, opened a new log
I20260924 21:12:35.617043 12374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:35.611541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43392 (local address 127.6.210.190:39473)
0924 21:12:35.612100 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:35.612111 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:35.612148 (+ 37us) server_negotiation.cc:407] Connection header received
0924 21:12:35.612656 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:35.612663 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:35.612724 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:35.612838 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:35.613066 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:35.613911 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:35.614989 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:35.615475 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:35.615567 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:35.616293 (+ 726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:35.616297 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:35.616297 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:12:35.616493 (+ 196us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:35.616534 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:12:35.616548 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:35.616622 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:35.616740 (+ 118us) server_negotiation.cc:299] Negotiation successful
0924 21:12:35.616823 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":443,"threads_started":1}
I20260924 21:12:35.618112 6986 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:12:35.618417 12371 raft_consensus.cc:348] T 00000000000000000000000000000000 P bdb7a424412f48338c916abb30057385 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdb7a424412f48338c916abb30057385" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39473 } }
I20260924 21:12:35.618588 12371 raft_consensus.cc:374] T 00000000000000000000000000000000 P bdb7a424412f48338c916abb30057385 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:35.618621 12371 raft_consensus.cc:729] T 00000000000000000000000000000000 P bdb7a424412f48338c916abb30057385 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdb7a424412f48338c916abb30057385, State: Initialized, Role: FOLLOWER
I20260924 21:12:35.618757 12371 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdb7a424412f48338c916abb30057385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdb7a424412f48338c916abb30057385" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39473 } }
I20260924 21:12:35.618835 12371 raft_consensus.cc:388] T 00000000000000000000000000000000 P bdb7a424412f48338c916abb30057385 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:12:35.618872 12371 raft_consensus.cc:482] T 00000000000000000000000000000000 P bdb7a424412f48338c916abb30057385 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:12:35.618937 12371 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bdb7a424412f48338c916abb30057385 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:35.619553 12371 raft_consensus.cc:504] T 00000000000000000000000000000000 P bdb7a424412f48338c916abb30057385 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdb7a424412f48338c916abb30057385" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39473 } }
I20260924 21:12:35.619645 12371 leader_election.cc:302] T 00000000000000000000000000000000 P bdb7a424412f48338c916abb30057385 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bdb7a424412f48338c916abb30057385; no voters:
I20260924 21:12:35.619757 12371 leader_election.cc:288] T 00000000000000000000000000000000 P bdb7a424412f48338c916abb30057385 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:12:35.619829 12376 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bdb7a424412f48338c916abb30057385 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:35.619993 12371 sys_catalog.cc:564] T 00000000000000000000000000000000 P bdb7a424412f48338c916abb30057385 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:12:35.620025 12376 raft_consensus.cc:686] T 00000000000000000000000000000000 P bdb7a424412f48338c916abb30057385 [term 1 LEADER]: Becoming Leader. State: Replica: bdb7a424412f48338c916abb30057385, State: Running, Role: LEADER
I20260924 21:12:35.620136 12376 consensus_queue.cc:237] T 00000000000000000000000000000000 P bdb7a424412f48338c916abb30057385 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdb7a424412f48338c916abb30057385" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39473 } }
I20260924 21:12:35.620522 12377 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdb7a424412f48338c916abb30057385 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bdb7a424412f48338c916abb30057385" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdb7a424412f48338c916abb30057385" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39473 } } }
I20260924 21:12:35.620541 12378 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdb7a424412f48338c916abb30057385 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdb7a424412f48338c916abb30057385. Latest consensus state: current_term: 1 leader_uuid: "bdb7a424412f48338c916abb30057385" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdb7a424412f48338c916abb30057385" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39473 } } }
I20260924 21:12:35.620675 12378 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdb7a424412f48338c916abb30057385 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:35.620831 12377 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdb7a424412f48338c916abb30057385 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:35.620908 12385 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:12:35.621446 12385 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:12:35.623065 12385 catalog_manager.cc:1347] Generated new cluster ID: fc6d13f0fc014bf9974c91bfa3a9e359
I20260924 21:12:35.623126 12385 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:12:35.674608 12385 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:12:35.675230 12385 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:12:35.682299 12385 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bdb7a424412f48338c916abb30057385: Generated new TSK 0
I20260924 21:12:35.682502 12385 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:12:35.692520 12374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:35.688525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43404 (local address 127.6.210.190:39473)
0924 21:12:35.688574 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:35.688576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:35.688691 (+ 115us) server_negotiation.cc:407] Connection header received
0924 21:12:35.688769 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:35.688770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:35.688784 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:35.688873 (+ 89us) server_negotiation.cc:226] Negotiated authn=JWT
0924 21:12:35.689006 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:35.689760 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:35.690772 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:35.691079 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:35.691160 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:35.691314 (+ 154us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0924 21:12:35.692293 (+ 979us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45081/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0924 21:12:35.692447 (+ 154us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43404: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45081/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":26}
W20260924 21:12:35.692589 12374 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43404: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45081/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260924 21:12:35.694376 12400 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:35.688447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39473 (local address 127.0.0.1:43404)
0924 21:12:35.688594 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:35.688604 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:35.688674 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:35.688878 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:35.688879 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:35.688881 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0924 21:12:35.688938 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:35.688940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:35.689863 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:35.689864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:35.690682 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:35.690685 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:35.691165 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:35.691166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:35.691212 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:35.691217 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0924 21:12:35.692403 (+ 1186us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45081/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0924 21:12:35.694261 (+ 1858us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.210.190:39473: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45081/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":1657,"thread_start_us":99,"threads_started":1}
I20260924 21:12:35.694859 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 12304
E20260924 21:12:35.704437 12401 webserver.cc:533] Webserver: sslize failed:
E20260924 21:12:35.704497 12401 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-24T21:12:35Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (256 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-24T21:12:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:12:35Z Disabled control of system clock
I20260924 21:12:35.741889 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:42903
--webserver_interface=127.6.210.190
--webserver_port=0
--builtin_ntp_servers=127.6.210.148:39477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:42903
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284333163253-6986-0/testchainca.cert with env {}
W20260924 21:12:35.820912 12405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:35.821113 12405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:35.821134 12405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:35.822343 12405 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:12:35.822383 12405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:35.822396 12405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:35.822408 12405 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:12:35.822464 12405 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260924 21:12:35.822484 12405 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260924 21:12:35.822510 12405 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260924 21:12:35.824007 12405 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:39477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:42903
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:42903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284333163253-6986-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:35.824265 12405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:35.824482 12405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:35.829190 12410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:35.829270 12413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:35.829293 12411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:35.829366 12405 server_base.cc:1034] running on GCE node
I20260924 21:12:35.829614 12405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:35.829902 12405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:35.831039 12405 hybrid_clock.cc:648] HybridClock initialized: now 1790284355831022 us; error 35 us; skew 500 ppm
I20260924 21:12:35.832224 12405 webserver.cc:460] Webserver started at http://127.6.210.190:42835/ using document root <none> and password file <none>
I20260924 21:12:35.832458 12405 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:35.832504 12405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:35.832664 12405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:35.833707 12405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "075e265ec0434ffb88cd329d461bc423"
format_stamp: "Formatted at 2026-09-24 21:12:35 on dist-test-slave-q276"
server_key: "f706456026677a54019d664ca001d8ac"
server_key_iv: "d651e06403b3f1df6a3846db221db0d7"
server_key_version: "encryptionkey@0"
I20260924 21:12:35.834066 12405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "075e265ec0434ffb88cd329d461bc423"
format_stamp: "Formatted at 2026-09-24 21:12:35 on dist-test-slave-q276"
server_key: "f706456026677a54019d664ca001d8ac"
server_key_iv: "d651e06403b3f1df6a3846db221db0d7"
server_key_version: "encryptionkey@0"
I20260924 21:12:35.835367 12405 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:35.836153 12419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:35.836337 12405 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:35.836402 12405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "075e265ec0434ffb88cd329d461bc423"
format_stamp: "Formatted at 2026-09-24 21:12:35 on dist-test-slave-q276"
server_key: "f706456026677a54019d664ca001d8ac"
server_key_iv: "d651e06403b3f1df6a3846db221db0d7"
server_key_version: "encryptionkey@0"
I20260924 21:12:35.836484 12405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:35.852243 12405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:35.852561 12405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:35.852717 12405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:35.856647 12405 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:42903
I20260924 21:12:35.856684 12471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:42903 every 8 connection(s)
I20260924 21:12:35.857062 12405 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:12:35.857963 12472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:35.860232 12472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 075e265ec0434ffb88cd329d461bc423: Bootstrap starting.
I20260924 21:12:35.860807 12472 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 075e265ec0434ffb88cd329d461bc423: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:35.861212 12472 log.cc:824] T 00000000000000000000000000000000 P 075e265ec0434ffb88cd329d461bc423: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:35.861953 12472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 075e265ec0434ffb88cd329d461bc423: No bootstrap required, opened a new log
I20260924 21:12:35.863775 12472 raft_consensus.cc:348] T 00000000000000000000000000000000 P 075e265ec0434ffb88cd329d461bc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "075e265ec0434ffb88cd329d461bc423" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 42903 } }
I20260924 21:12:35.863932 12472 raft_consensus.cc:374] T 00000000000000000000000000000000 P 075e265ec0434ffb88cd329d461bc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:35.863998 12472 raft_consensus.cc:729] T 00000000000000000000000000000000 P 075e265ec0434ffb88cd329d461bc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 075e265ec0434ffb88cd329d461bc423, State: Initialized, Role: FOLLOWER
I20260924 21:12:35.864167 12472 consensus_queue.cc:260] T 00000000000000000000000000000000 P 075e265ec0434ffb88cd329d461bc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "075e265ec0434ffb88cd329d461bc423" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 42903 } }
I20260924 21:12:35.864236 12472 raft_consensus.cc:388] T 00000000000000000000000000000000 P 075e265ec0434ffb88cd329d461bc423 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:12:35.864280 12472 raft_consensus.cc:482] T 00000000000000000000000000000000 P 075e265ec0434ffb88cd329d461bc423 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:12:35.864336 12472 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 075e265ec0434ffb88cd329d461bc423 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:35.864949 12472 raft_consensus.cc:504] T 00000000000000000000000000000000 P 075e265ec0434ffb88cd329d461bc423 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "075e265ec0434ffb88cd329d461bc423" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 42903 } }
I20260924 21:12:35.865079 12472 leader_election.cc:302] T 00000000000000000000000000000000 P 075e265ec0434ffb88cd329d461bc423 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 075e265ec0434ffb88cd329d461bc423; no voters:
I20260924 21:12:35.865280 12472 leader_election.cc:288] T 00000000000000000000000000000000 P 075e265ec0434ffb88cd329d461bc423 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:12:35.865303 12475 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 075e265ec0434ffb88cd329d461bc423 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:35.865451 12475 raft_consensus.cc:686] T 00000000000000000000000000000000 P 075e265ec0434ffb88cd329d461bc423 [term 1 LEADER]: Becoming Leader. State: Replica: 075e265ec0434ffb88cd329d461bc423, State: Running, Role: LEADER
I20260924 21:12:35.865561 12475 consensus_queue.cc:237] T 00000000000000000000000000000000 P 075e265ec0434ffb88cd329d461bc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "075e265ec0434ffb88cd329d461bc423" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 42903 } }
I20260924 21:12:35.865599 12472 sys_catalog.cc:564] T 00000000000000000000000000000000 P 075e265ec0434ffb88cd329d461bc423 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:12:35.865942 12477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 075e265ec0434ffb88cd329d461bc423 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 075e265ec0434ffb88cd329d461bc423. Latest consensus state: current_term: 1 leader_uuid: "075e265ec0434ffb88cd329d461bc423" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "075e265ec0434ffb88cd329d461bc423" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 42903 } } }
I20260924 21:12:35.866036 12477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 075e265ec0434ffb88cd329d461bc423 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:35.866102 12476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 075e265ec0434ffb88cd329d461bc423 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "075e265ec0434ffb88cd329d461bc423" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "075e265ec0434ffb88cd329d461bc423" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 42903 } } }
I20260924 21:12:35.866169 12476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 075e265ec0434ffb88cd329d461bc423 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:35.866371 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 12405
I20260924 21:12:35.866508 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:12:35.866708 6986 external_mini_cluster.cc:1428] Setting key dd2c6f4a0c4d507e2bb74c668a2bf286
I20260924 21:12:35.867074 12481 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:12:35.867564 12481 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:12:35.869220 12481 catalog_manager.cc:1347] Generated new cluster ID: fdd8dd85b79c4be592b48063834ebbc9
I20260924 21:12:35.869277 12481 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:12:35.873533 12488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:35.867354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58816 (local address 127.6.210.190:42903)
0924 21:12:35.868171 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:35.868182 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:35.868220 (+ 38us) server_negotiation.cc:407] Connection header received
0924 21:12:35.868341 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:35.868349 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:35.868420 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:35.868580 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:35.868816 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:35.870184 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:35.871399 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:35.872004 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:35.872104 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:35.872839 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:35.872842 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:35.872843 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:12:35.873035 (+ 192us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:35.873073 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:12:35.873085 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:35.873145 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:35.873266 (+ 121us) server_negotiation.cc:299] Negotiation successful
0924 21:12:35.873338 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":71,"threads_started":1}
I20260924 21:12:35.874522 6986 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:12:35.875542 12481 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:12:35.875975 12481 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:12:35.879514 12481 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 075e265ec0434ffb88cd329d461bc423: Generated new TSK 0
I20260924 21:12:35.879678 12481 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:12:36.324071 12488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:35.883942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58832 (local address 127.6.210.190:42903)
0924 21:12:35.883984 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:35.883986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:35.884081 (+ 95us) server_negotiation.cc:407] Connection header received
0924 21:12:35.884168 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:35.884168 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:35.884178 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:35.884233 (+ 55us) server_negotiation.cc:226] Negotiated authn=JWT
0924 21:12:35.884383 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:35.884830 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:35.886058 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:35.886226 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:35.886320 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:35.886498 (+ 178us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0924 21:12:36.323850 (+437352us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0924 21:12:36.324013 (+ 163us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58832: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":22}
W20260924 21:12:36.324182 12488 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58832: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260924 21:12:36.324290 12501 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:35.883864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:42903 (local address 127.0.0.1:58832)
0924 21:12:35.884020 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:35.884032 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:35.884099 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:35.884254 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:35.884255 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:35.884257 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0924 21:12:35.884319 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:35.884321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:35.884912 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:35.884914 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:35.885966 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:35.885969 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:35.886333 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:35.886335 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:35.886400 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:35.886406 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0924 21:12:36.324019 (+437613us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0924 21:12:36.324172 (+ 153us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.210.190:42903: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":87,"threads_started":1}
I20260924 21:12:36.324661 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 12405
2026-09-24T21:12:36Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (607 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (88 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (87 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:12:36 dist-test-slave-q276 krb5kdc[12513](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:12:36 dist-test-slave-q276 krb5kdc[12513](info): set up 2 sockets
Sep 24 21:12:36 dist-test-slave-q276 krb5kdc[12513](info): commencing operation
krb5kdc: starting...
W20260924 21:12:37.639456 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:12:37 dist-test-slave-q276 krb5kdc[12513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:12:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:12:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.210.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
I20260924 21:12:37.761607 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:34365
--webserver_interface=127.6.210.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:43407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:34365
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:37.841217 12526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:37.841418 12526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:37.841439 12526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:37.842644 12526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:12:37.842689 12526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:37.842703 12526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:37.842746 12526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:12:37.842775 12526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:12:37.844233 12526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:43407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:34365
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:34365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:37.844488 12526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:37.844722 12526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 21:12:37.849854 12526 server_base.cc:1034] running on GCE node
W20260924 21:12:37.849838 12531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:37.849838 12534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:37.849828 12532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:37.850446 12526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:37.850759 12526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:37.851926 12526 hybrid_clock.cc:648] HybridClock initialized: now 1790284357851921 us; error 35 us; skew 500 ppm
Sep 24 21:12:37 dist-test-slave-q276 krb5kdc[12513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284357, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:37.854451 12526 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:12:37.854934 12526 webserver.cc:460] Webserver started at http://127.6.210.190:33689/ using document root <none> and password file <none>
I20260924 21:12:37.855170 12526 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:37.855238 12526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:37.855388 12526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:37.856431 12526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "a645a700714e45488a863d7fd66b6730"
format_stamp: "Formatted at 2026-09-24 21:12:37 on dist-test-slave-q276"
server_key: "ec1b116221a3a85a6475a0f2036cd970"
server_key_iv: "e9084c88a16c06498707e94d23045d7b"
server_key_version: "encryptionkey@0"
I20260924 21:12:37.856802 12526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "a645a700714e45488a863d7fd66b6730"
format_stamp: "Formatted at 2026-09-24 21:12:37 on dist-test-slave-q276"
server_key: "ec1b116221a3a85a6475a0f2036cd970"
server_key_iv: "e9084c88a16c06498707e94d23045d7b"
server_key_version: "encryptionkey@0"
I20260924 21:12:37.858170 12526 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:12:37.858912 12541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:37.859081 12526 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:12:37.859172 12526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "a645a700714e45488a863d7fd66b6730"
format_stamp: "Formatted at 2026-09-24 21:12:37 on dist-test-slave-q276"
server_key: "ec1b116221a3a85a6475a0f2036cd970"
server_key_iv: "e9084c88a16c06498707e94d23045d7b"
server_key_version: "encryptionkey@0"
I20260924 21:12:37.859254 12526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:37.871591 12526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:37.873965 12526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:37.874132 12526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:37.878016 12526 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:34365
I20260924 21:12:37.878050 12593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:34365 every 8 connection(s)
I20260924 21:12:37.878485 12526 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:12:37.879341 12594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:37.881536 12594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a645a700714e45488a863d7fd66b6730: Bootstrap starting.
I20260924 21:12:37.882099 12594 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a645a700714e45488a863d7fd66b6730: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:37.882424 12594 log.cc:824] T 00000000000000000000000000000000 P a645a700714e45488a863d7fd66b6730: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:37.883152 12594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a645a700714e45488a863d7fd66b6730: No bootstrap required, opened a new log
I20260924 21:12:37.884670 12594 raft_consensus.cc:348] T 00000000000000000000000000000000 P a645a700714e45488a863d7fd66b6730 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a645a700714e45488a863d7fd66b6730" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34365 } }
I20260924 21:12:37.884800 12594 raft_consensus.cc:374] T 00000000000000000000000000000000 P a645a700714e45488a863d7fd66b6730 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:37.884822 12594 raft_consensus.cc:729] T 00000000000000000000000000000000 P a645a700714e45488a863d7fd66b6730 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a645a700714e45488a863d7fd66b6730, State: Initialized, Role: FOLLOWER
I20260924 21:12:37.884972 12594 consensus_queue.cc:260] T 00000000000000000000000000000000 P a645a700714e45488a863d7fd66b6730 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a645a700714e45488a863d7fd66b6730" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34365 } }
I20260924 21:12:37.885032 12594 raft_consensus.cc:388] T 00000000000000000000000000000000 P a645a700714e45488a863d7fd66b6730 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:12:37.885073 12594 raft_consensus.cc:482] T 00000000000000000000000000000000 P a645a700714e45488a863d7fd66b6730 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:12:37.885133 12594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a645a700714e45488a863d7fd66b6730 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:37.885625 12594 raft_consensus.cc:504] T 00000000000000000000000000000000 P a645a700714e45488a863d7fd66b6730 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a645a700714e45488a863d7fd66b6730" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34365 } }
I20260924 21:12:37.885735 12594 leader_election.cc:302] T 00000000000000000000000000000000 P a645a700714e45488a863d7fd66b6730 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a645a700714e45488a863d7fd66b6730; no voters:
I20260924 21:12:37.885911 12594 leader_election.cc:288] T 00000000000000000000000000000000 P a645a700714e45488a863d7fd66b6730 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:12:37.885955 12597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a645a700714e45488a863d7fd66b6730 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:37.886184 12594 sys_catalog.cc:564] T 00000000000000000000000000000000 P a645a700714e45488a863d7fd66b6730 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:12:37.886251 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 12526
I20260924 21:12:37.886415 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:12:37.886641 6986 external_mini_cluster.cc:1428] Setting key c6313b480b8982704e5f8ad82946f35a
I20260924 21:12:37.886857 12597 raft_consensus.cc:686] T 00000000000000000000000000000000 P a645a700714e45488a863d7fd66b6730 [term 1 LEADER]: Becoming Leader. State: Replica: a645a700714e45488a863d7fd66b6730, State: Running, Role: LEADER
I20260924 21:12:37.886969 12597 consensus_queue.cc:237] T 00000000000000000000000000000000 P a645a700714e45488a863d7fd66b6730 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a645a700714e45488a863d7fd66b6730" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34365 } }
I20260924 21:12:37.887558 12598 sys_catalog.cc:455] T 00000000000000000000000000000000 P a645a700714e45488a863d7fd66b6730 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a645a700714e45488a863d7fd66b6730" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a645a700714e45488a863d7fd66b6730" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34365 } } }
I20260924 21:12:37.887660 12598 sys_catalog.cc:458] T 00000000000000000000000000000000 P a645a700714e45488a863d7fd66b6730 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:37.887944 12603 sys_catalog.cc:455] T 00000000000000000000000000000000 P a645a700714e45488a863d7fd66b6730 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a645a700714e45488a863d7fd66b6730. Latest consensus state: current_term: 1 leader_uuid: "a645a700714e45488a863d7fd66b6730" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a645a700714e45488a863d7fd66b6730" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34365 } } }
I20260924 21:12:37.888036 12603 sys_catalog.cc:458] T 00000000000000000000000000000000 P a645a700714e45488a863d7fd66b6730 [sys.catalog]: This master's current role is: LEADER
W20260924 21:12:37.888301 12613 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a645a700714e45488a863d7fd66b6730: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 21:12:37.888363 12613 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 21:12:37.888422 12614 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:12:37.888621 12614 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:12:37.890345 12614 catalog_manager.cc:1347] Generated new cluster ID: 21e9d42d5ac4463295185cf88ea5659a
I20260924 21:12:37.890403 12614 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 21:12:37 dist-test-slave-q276 krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:37.899902 12609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:37.887315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32840 (local address 127.6.210.190:34365)
0924 21:12:37.888104 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:37.888114 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:37.888149 (+ 35us) server_negotiation.cc:407] Connection header received
0924 21:12:37.888261 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:37.888267 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:37.888310 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:37.888505 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:37.889335 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:37.890555 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:37.891698 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:37.892084 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:37.892166 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:37.895119 (+ 2953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:37.895123 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:37.895123 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:37.895136 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:37.897932 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:37.898398 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:37.898401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:37.898401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:37.898529 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:37.898769 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:37.898770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:37.898770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:37.899099 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:37.899297 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:37.899528 (+ 231us) server_negotiation.cc:299] Negotiation successful
0924 21:12:37.899684 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":737,"thread_start_us":74,"threads_started":1}
I20260924 21:12:37.908818 12614 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:12:37.909547 12614 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:12:37.916991 12614 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a645a700714e45488a863d7fd66b6730: Generated new TSK 0
I20260924 21:12:37.917228 12614 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.210.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
I20260924 21:12:37.947152 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.129:0
--local_ip_for_outbound_sockets=127.6.210.129
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:34365
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:43407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:38.027757 12620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:38.027967 12620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:38.027989 12620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:38.029146 12620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:38.029189 12620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:38.029217 12620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.129
I20260924 21:12:38.030754 12620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:43407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:34365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.129
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:38.031009 12620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:38.031255 12620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:38.036437 12625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:38.036499 12628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:38.036437 12626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:38.036739 12620 server_base.cc:1034] running on GCE node
I20260924 21:12:38.036959 12620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:38.037199 12620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:38.038321 12620 hybrid_clock.cc:648] HybridClock initialized: now 1790284358038305 us; error 27 us; skew 500 ppm
Sep 24 21:12:38 dist-test-slave-q276 krb5kdc[12513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284358, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:38.040797 12620 init.cc:377] Logged in from keytab as kudu/127.6.210.129@KRBTEST.COM (short username kudu)
I20260924 21:12:38.041270 12620 webserver.cc:460] Webserver started at http://127.6.210.129:44587/ using document root <none> and password file <none>
I20260924 21:12:38.041509 12620 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:38.041575 12620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:38.041715 12620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:38.042819 12620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-0/data/instance:
uuid: "b185121223a948709243c8aefb1b5616"
format_stamp: "Formatted at 2026-09-24 21:12:38 on dist-test-slave-q276"
server_key: "b6d98bbc9cdce99c0ceae8ed7793908a"
server_key_iv: "ba9c55594ed82dcb5b90ef2d4697a3e5"
server_key_version: "encryptionkey@0"
I20260924 21:12:38.043185 12620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance:
uuid: "b185121223a948709243c8aefb1b5616"
format_stamp: "Formatted at 2026-09-24 21:12:38 on dist-test-slave-q276"
server_key: "b6d98bbc9cdce99c0ceae8ed7793908a"
server_key_iv: "ba9c55594ed82dcb5b90ef2d4697a3e5"
server_key_version: "encryptionkey@0"
I20260924 21:12:38.044487 12620 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:12:38.045303 12635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:38.045501 12620 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:38.045598 12620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-0/wal
uuid: "b185121223a948709243c8aefb1b5616"
format_stamp: "Formatted at 2026-09-24 21:12:38 on dist-test-slave-q276"
server_key: "b6d98bbc9cdce99c0ceae8ed7793908a"
server_key_iv: "ba9c55594ed82dcb5b90ef2d4697a3e5"
server_key_version: "encryptionkey@0"
I20260924 21:12:38.045692 12620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:38.077466 12620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:38.079433 12620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:38.079618 12620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:38.079870 12620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:38.080195 12620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:38.080250 12620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:38.080286 12620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:38.080313 12620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:38.086179 12620 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.129:37991
I20260924 21:12:38.086228 12748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.129:37991 every 8 connection(s)
I20260924 21:12:38.086588 12620 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
Sep 24 21:12:38 dist-test-slave-q276 krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284358, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:38.092559 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 12620
I20260924 21:12:38.092710 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance
I20260924 21:12:38.092892 6986 external_mini_cluster.cc:1428] Setting key 9cf3a196b6f6c3b626c0c2c75db9baa0
I20260924 21:12:38.096810 12609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:38.087387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:57495 (local address 127.6.210.190:34365)
0924 21:12:38.087458 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:38.087460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:38.087738 (+ 278us) server_negotiation.cc:407] Connection header received
0924 21:12:38.088716 (+ 978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:38.088717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:38.088730 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:38.088787 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:38.089358 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:38.089853 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:38.090818 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:38.091002 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:38.091054 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:38.093222 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:38.093224 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:38.093224 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:38.093232 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:38.095096 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:38.095651 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:38.095652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:38.095652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:38.095705 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:38.095993 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:38.095993 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:38.095993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:38.096176 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:38.096248 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:38.096597 (+ 349us) server_negotiation.cc:299] Negotiation successful
0924 21:12:38.096714 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260924 21:12:38.097018 12751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:38.087374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34365 (local address 127.6.210.129:57495)
0924 21:12:38.087581 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:38.087642 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:38.088615 (+ 973us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:38.088844 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:38.088847 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:38.089096 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:38.089282 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:38.089285 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:38.089929 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:38.089929 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:38.090741 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:38.090743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:38.091073 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:38.091074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:38.091169 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:38.091639 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:38.091647 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:38.093089 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:38.095203 (+ 2114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:38.095212 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:38.095214 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:38.095554 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:38.095776 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:38.095777 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:38.095777 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:38.095927 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:38.096257 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:38.096266 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:38.096446 (+ 180us) client_negotiation.cc:769] Sending connection context
0924 21:12:38.096570 (+ 124us) client_negotiation.cc:240] Negotiation successful
0924 21:12:38.096771 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260924 21:12:38.097733 12749 heartbeater.cc:347] Connected to a master server at 127.6.210.190:34365
I20260924 21:12:38.097855 12749 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:38.098031 12749 heartbeater.cc:511] Master 127.6.210.190:34365 requested a full tablet report, sending...
I20260924 21:12:38.098625 12558 ts_manager.cc:194] Registered new tserver with Master: b185121223a948709243c8aefb1b5616 (127.6.210.129:37991)
I20260924 21:12:38.099615 12558 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.129@KRBTEST.COM'} at 127.6.210.129:57495
WARNING: no policy specified for kudu/127.6.210.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
I20260924 21:12:38.119385 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.130:0
--local_ip_for_outbound_sockets=127.6.210.130
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:34365
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:43407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:38.200378 12754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:38.200613 12754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:38.200650 12754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:38.201792 12754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:38.201859 12754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:38.201902 12754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.130
I20260924 21:12:38.203378 12754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:43407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:34365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.130
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:38.203642 12754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:38.203879 12754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:38.208889 12762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:38.208892 12760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:38.208945 12759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:38.208966 12754 server_base.cc:1034] running on GCE node
I20260924 21:12:38.209395 12754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:38.209635 12754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:38.210791 12754 hybrid_clock.cc:648] HybridClock initialized: now 1790284358210782 us; error 39 us; skew 500 ppm
Sep 24 21:12:38 dist-test-slave-q276 krb5kdc[12513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284358, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:38.213479 12754 init.cc:377] Logged in from keytab as kudu/127.6.210.130@KRBTEST.COM (short username kudu)
I20260924 21:12:38.213951 12754 webserver.cc:460] Webserver started at http://127.6.210.130:44925/ using document root <none> and password file <none>
I20260924 21:12:38.214185 12754 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:38.214253 12754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:38.214434 12754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:38.215482 12754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-1/data/instance:
uuid: "a5e8eaf4b0d147a38b6dd68f88102ea3"
format_stamp: "Formatted at 2026-09-24 21:12:38 on dist-test-slave-q276"
server_key: "6f91a792faf825140ae47fc0ec5d356f"
server_key_iv: "973fe68b58306ddc0a34220690efc854"
server_key_version: "encryptionkey@0"
I20260924 21:12:38.215859 12754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance:
uuid: "a5e8eaf4b0d147a38b6dd68f88102ea3"
format_stamp: "Formatted at 2026-09-24 21:12:38 on dist-test-slave-q276"
server_key: "6f91a792faf825140ae47fc0ec5d356f"
server_key_iv: "973fe68b58306ddc0a34220690efc854"
server_key_version: "encryptionkey@0"
I20260924 21:12:38.217150 12754 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:12:38.217849 12769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:38.218004 12754 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:12:38.218099 12754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-1/wal
uuid: "a5e8eaf4b0d147a38b6dd68f88102ea3"
format_stamp: "Formatted at 2026-09-24 21:12:38 on dist-test-slave-q276"
server_key: "6f91a792faf825140ae47fc0ec5d356f"
server_key_iv: "973fe68b58306ddc0a34220690efc854"
server_key_version: "encryptionkey@0"
I20260924 21:12:38.218197 12754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:38.227695 12754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:38.230033 12754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:38.230195 12754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:38.230530 12754 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:38.230832 12754 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:38.230886 12754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:38.230926 12754 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:38.230962 12754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:38.236881 12754 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.130:35089
I20260924 21:12:38.236935 12882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.130:35089 every 8 connection(s)
I20260924 21:12:38.237253 12754 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
Sep 24 21:12:38 dist-test-slave-q276 krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284358, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:38.244299 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 12754
I20260924 21:12:38.244407 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance
I20260924 21:12:38.244604 6986 external_mini_cluster.cc:1428] Setting key 45bb8db8d0d20f3e20ce55eac6771f45
I20260924 21:12:38.246851 12609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:38.238037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:53847 (local address 127.6.210.190:34365)
0924 21:12:38.238113 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:38.238115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:38.238267 (+ 152us) server_negotiation.cc:407] Connection header received
0924 21:12:38.239227 (+ 960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:38.239227 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:38.239238 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:38.239306 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:38.239864 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:38.240402 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:38.241339 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:38.241533 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:38.241580 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:38.243746 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:38.243747 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:38.243747 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:38.243753 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:38.245436 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:38.245831 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:38.245831 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:38.245832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:38.245892 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:38.246113 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:38.246114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:38.246114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:38.246274 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:38.246339 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:38.246635 (+ 296us) server_negotiation.cc:299] Negotiation successful
0924 21:12:38.246731 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260924 21:12:38.247054 12885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:38.237998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34365 (local address 127.6.210.130:53847)
0924 21:12:38.238139 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:38.238179 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:38.239134 (+ 955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:38.239361 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:38.239364 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:38.239609 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:38.239793 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:38.239796 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:38.240481 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:38.240482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:38.241258 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:38.241260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:38.241610 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:38.241611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:38.241711 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:38.242239 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:38.242247 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:38.243657 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:38.245512 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:38.245517 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:38.245519 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:38.245756 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:38.245958 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:38.245959 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:38.245959 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:38.246053 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:38.246351 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:38.246357 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:38.246500 (+ 143us) client_negotiation.cc:769] Sending connection context
0924 21:12:38.246586 (+ 86us) client_negotiation.cc:240] Negotiation successful
0924 21:12:38.246780 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":76,"threads_started":1}
I20260924 21:12:38.247540 12883 heartbeater.cc:347] Connected to a master server at 127.6.210.190:34365
I20260924 21:12:38.247655 12883 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:38.247828 12883 heartbeater.cc:511] Master 127.6.210.190:34365 requested a full tablet report, sending...
I20260924 21:12:38.248219 12558 ts_manager.cc:194] Registered new tserver with Master: a5e8eaf4b0d147a38b6dd68f88102ea3 (127.6.210.130:35089)
I20260924 21:12:38.248740 12558 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.130@KRBTEST.COM'} at 127.6.210.130:53847
WARNING: no policy specified for kudu/127.6.210.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
I20260924 21:12:38.269755 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.131:0
--local_ip_for_outbound_sockets=127.6.210.131
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:34365
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:43407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:38.350162 12888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:38.350375 12888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:38.350399 12888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:38.351480 12888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:38.351513 12888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:38.351538 12888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.131
I20260924 21:12:38.352928 12888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:43407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:34365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.131
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:38.353108 12888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:38.353273 12888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:38.358584 12894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:38.358593 12893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:38.358593 12896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:38.358796 12888 server_base.cc:1034] running on GCE node
I20260924 21:12:38.359159 12888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:38.359405 12888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:38.360592 12888 hybrid_clock.cc:648] HybridClock initialized: now 1790284358360573 us; error 47 us; skew 500 ppm
Sep 24 21:12:38 dist-test-slave-q276 krb5kdc[12513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284358, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:38.363207 12888 init.cc:377] Logged in from keytab as kudu/127.6.210.131@KRBTEST.COM (short username kudu)
I20260924 21:12:38.363678 12888 webserver.cc:460] Webserver started at http://127.6.210.131:44535/ using document root <none> and password file <none>
I20260924 21:12:38.363917 12888 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:38.363987 12888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:38.364154 12888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:38.365222 12888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-2/data/instance:
uuid: "63540fe63b0d4c898f0443a50987eab4"
format_stamp: "Formatted at 2026-09-24 21:12:38 on dist-test-slave-q276"
server_key: "f251037e42461177e95eebc59458eed2"
server_key_iv: "ecad2e68c81deb4688cea97f89e70f30"
server_key_version: "encryptionkey@0"
I20260924 21:12:38.365633 12888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance:
uuid: "63540fe63b0d4c898f0443a50987eab4"
format_stamp: "Formatted at 2026-09-24 21:12:38 on dist-test-slave-q276"
server_key: "f251037e42461177e95eebc59458eed2"
server_key_iv: "ecad2e68c81deb4688cea97f89e70f30"
server_key_version: "encryptionkey@0"
I20260924 21:12:38.366999 12888 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:12:38.367714 12903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:38.367877 12888 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:38.367973 12888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-2/wal
uuid: "63540fe63b0d4c898f0443a50987eab4"
format_stamp: "Formatted at 2026-09-24 21:12:38 on dist-test-slave-q276"
server_key: "f251037e42461177e95eebc59458eed2"
server_key_iv: "ecad2e68c81deb4688cea97f89e70f30"
server_key_version: "encryptionkey@0"
I20260924 21:12:38.368048 12888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:38.376151 12888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:38.377915 12888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:38.378068 12888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:38.378373 12888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:38.378670 12888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:38.378724 12888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:38.378764 12888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:38.378790 12888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:38.384572 12888 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.131:44861
I20260924 21:12:38.384626 13016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.131:44861 every 8 connection(s)
I20260924 21:12:38.384959 12888 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
Sep 24 21:12:38 dist-test-slave-q276 krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284358, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:38.394038 12609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:38.385706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:34121 (local address 127.6.210.190:34365)
0924 21:12:38.385822 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:38.385824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:38.385985 (+ 161us) server_negotiation.cc:407] Connection header received
0924 21:12:38.387052 (+ 1067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:38.387053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:38.387065 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:38.387146 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:38.387712 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:38.388339 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:38.389309 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:38.389492 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:38.389554 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:38.391557 (+ 2003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:38.391558 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:38.391559 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:38.391565 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:38.392966 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:38.393312 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:38.393313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:38.393313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:38.393360 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:38.393558 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:38.393558 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:38.393558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:38.393654 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:38.393704 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:38.393918 (+ 214us) server_negotiation.cc:299] Negotiation successful
0924 21:12:38.393975 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260924 21:12:38.394251 13019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:38.385695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34365 (local address 127.6.210.131:34121)
0924 21:12:38.385861 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:38.385902 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:38.386947 (+ 1045us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:38.387199 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:38.387202 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:38.387443 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:38.387628 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:38.387632 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:38.388423 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:38.388424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:38.389220 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:38.389222 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:38.389568 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:38.389568 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:38.389662 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:38.390120 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:38.390128 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:38.391467 (+ 1339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:38.393032 (+ 1565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:38.393037 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:38.393038 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:38.393249 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:38.393416 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:38.393417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:38.393417 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:38.393505 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:38.393707 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:38.393711 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:38.393813 (+ 102us) client_negotiation.cc:769] Sending connection context
0924 21:12:38.393891 (+ 78us) client_negotiation.cc:240] Negotiation successful
0924 21:12:38.393998 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":72,"threads_started":1}
I20260924 21:12:38.394717 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 12888
I20260924 21:12:38.394822 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance
I20260924 21:12:38.394807 13017 heartbeater.cc:347] Connected to a master server at 127.6.210.190:34365
I20260924 21:12:38.394894 13017 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:38.395020 6986 external_mini_cluster.cc:1428] Setting key d87b2954686c3b5dc374c1efbe72c4f8
I20260924 21:12:38.395071 13017 heartbeater.cc:511] Master 127.6.210.190:34365 requested a full tablet report, sending...
I20260924 21:12:38.395449 12558 ts_manager.cc:194] Registered new tserver with Master: 63540fe63b0d4c898f0443a50987eab4 (127.6.210.131:44861)
I20260924 21:12:38.395880 6986 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:12:38.395884 12558 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.131@KRBTEST.COM'} at 127.6.210.131:34121
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/test-admin.keytab.
Sep 24 21:12:38 dist-test-slave-q276 krb5kdc[12513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:38.410656 6986 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260924 21:12:38.412314 13024 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:38.411567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35268 (local address 127.6.210.129:37991)
0924 21:12:38.411752 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:38.411758 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:38.411770 (+ 12us) server_negotiation.cc:407] Connection header received
0924 21:12:38.411805 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:38.411806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:38.411818 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:38.411890 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:38.412232 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35268: BlockingRecv error: recv got EOF from 127.0.0.1:35268 (error 108)
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":102,"threads_started":1}
W20260924 21:12:38.412446 6986 init.cc:244] Error while opening credential cache '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/krb5cc)
Sep 24 21:12:38 dist-test-slave-q276 krb5kdc[12513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:38.413568 6986 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 24 21:12:38 dist-test-slave-q276 krb5kdc[12513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:38.414577 6986 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 24 21:12:38 dist-test-slave-q276 krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.129@KRBTEST.COM
I20260924 21:12:38.422873 13024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:38.415134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35284 (local address 127.6.210.129:37991)
0924 21:12:38.415200 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:38.415202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:38.415260 (+ 58us) server_negotiation.cc:407] Connection header received
0924 21:12:38.415347 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:38.415348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:38.415360 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:38.415432 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:38.415819 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:38.416516 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:38.417337 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:38.417691 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:38.417768 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:38.419894 (+ 2126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:38.419902 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:38.419903 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:38.419909 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:38.421642 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:38.422019 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:38.422023 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:38.422023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:38.422078 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:38.422357 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:38.422358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:38.422358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:38.422493 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:38.422620 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:38.422728 (+ 108us) server_negotiation.cc:299] Negotiation successful
0924 21:12:38.422798 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260924 21:12:38.424753 13024 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:38.424161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35286 (local address 127.6.210.129:37991)
0924 21:12:38.424212 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:38.424214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:38.424316 (+ 102us) server_negotiation.cc:407] Connection header received
0924 21:12:38.424351 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:38.424352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:38.424361 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:38.424404 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:38.424677 (+ 273us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35286: BlockingRecv error: recv got EOF from 127.0.0.1:35286 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260924 21:12:38.424829 6986 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/krb5cc' of type FILE
Sep 24 21:12:38 dist-test-slave-q276 krb5kdc[12513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:38.425768 6986 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 24 21:12:38 dist-test-slave-q276 krb5kdc[12513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:38.426687 6986 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 24 21:12:38 dist-test-slave-q276 krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.129@KRBTEST.COM
I20260924 21:12:38.433218 13024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:38.427147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35292 (local address 127.6.210.129:37991)
0924 21:12:38.427200 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:38.427202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:38.427258 (+ 56us) server_negotiation.cc:407] Connection header received
0924 21:12:38.427295 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:38.427296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:38.427305 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:38.427349 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:38.427652 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:38.428210 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:38.428862 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:38.429042 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:38.429089 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:38.431072 (+ 1983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:38.431073 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:38.431073 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:38.431078 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:38.432252 (+ 1174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:38.432538 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:38.432538 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:38.432539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:38.432585 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:38.432743 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:38.432743 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:38.432744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:38.432818 (+ 74us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:38.432883 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:38.433002 (+ 119us) server_negotiation.cc:299] Negotiation successful
0924 21:12:38.433056 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260924 21:12:38.434732 13024 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:38.434165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35306 (local address 127.6.210.129:37991)
0924 21:12:38.434230 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:38.434232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:38.434241 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:12:38.434356 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:38.434357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:38.434367 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:38.434419 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:38.434683 (+ 264us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35306: BlockingRecv error: recv got EOF from 127.0.0.1:35306 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260924 21:12:38.434872 6986 init.cc:244] Error while opening credential cache '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284333163253-6986-0/krb5kdc/krb5cc)
Sep 24 21:12:38 dist-test-slave-q276 krb5kdc[12513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:38.435776 6986 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 24 21:12:38 dist-test-slave-q276 krb5kdc[12513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:38.436604 6986 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 24 21:12:38 dist-test-slave-q276 krb5kdc[12513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.129@KRBTEST.COM
I20260924 21:12:38.443151 13024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:38.437096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35314 (local address 127.6.210.129:37991)
0924 21:12:38.437167 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:38.437168 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:38.437201 (+ 33us) server_negotiation.cc:407] Connection header received
0924 21:12:38.437247 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:38.437248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:38.437255 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:38.437306 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:38.437622 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:38.438071 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:38.438782 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:38.438954 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:38.439007 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:38.440891 (+ 1884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:38.440892 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:38.440892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:38.440898 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:38.442324 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:38.442620 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:38.442621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:38.442621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:38.442670 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:38.442811 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:38.442812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:38.442812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:38.442891 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:38.442953 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:38.443037 (+ 84us) server_negotiation.cc:299] Negotiation successful
0924 21:12:38.443090 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 21:12:38.443852 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 12620
I20260924 21:12:38.448333 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 12754
I20260924 21:12:38.452581 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 12888
I20260924 21:12:38.456813 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 12526
2026-09-24T21:12:38Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1957 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:12:38 dist-test-slave-q276 krb5kdc[13037](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:12:38 dist-test-slave-q276 krb5kdc[13037](info): set up 2 sockets
Sep 24 21:12:38 dist-test-slave-q276 krb5kdc[13037](info): commencing operation
krb5kdc: starting...
W20260924 21:12:39.503532 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:12:39 dist-test-slave-q276 krb5kdc[13037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:12:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:12:39Z Disabled control of system clock
WARNING: no policy specified for oryx/127.6.210.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.210.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.210.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.190.keytab.
Entry for principal oryx/127.6.210.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.190.keytab.
I20260924 21:12:39.624162 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:39481
--webserver_interface=127.6.210.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.190.keytab
--principal=oryx/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:39813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:39481
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:39.703876 13050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:39.704084 13050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:39.704106 13050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:39.705266 13050 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:12:39.705307 13050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:39.705322 13050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:39.705334 13050 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:12:39.705346 13050 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:12:39.706831 13050 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:39813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:39481
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.190.keytab
--principal=oryx/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:39481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:39.707079 13050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:39.707341 13050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:39.712038 13055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:39.712066 13056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:39.712072 13058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:39.712149 13050 server_base.cc:1034] running on GCE node
I20260924 21:12:39.712438 13050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:39.712760 13050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:39.713920 13050 hybrid_clock.cc:648] HybridClock initialized: now 1790284359713906 us; error 27 us; skew 500 ppm
Sep 24 21:12:39 dist-test-slave-q276 krb5kdc[13037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284359, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:39.720000 13050 init.cc:377] Logged in from keytab as oryx/127.6.210.190@KRBTEST.COM (short username oryx)
I20260924 21:12:39.720503 13050 webserver.cc:460] Webserver started at http://127.6.210.190:39447/ using document root <none> and password file <none>
I20260924 21:12:39.720741 13050 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:39.720808 13050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:39.720961 13050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:39.722000 13050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "34863bba7e8146cab737678d7499c335"
format_stamp: "Formatted at 2026-09-24 21:12:39 on dist-test-slave-q276"
server_key: "5a58c1ee2441304b88e35d6a64c8e5c2"
server_key_iv: "407da9cc88db3b89a8dc67a8df1ad32b"
server_key_version: "encryptionkey@0"
I20260924 21:12:39.722404 13050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "34863bba7e8146cab737678d7499c335"
format_stamp: "Formatted at 2026-09-24 21:12:39 on dist-test-slave-q276"
server_key: "5a58c1ee2441304b88e35d6a64c8e5c2"
server_key_iv: "407da9cc88db3b89a8dc67a8df1ad32b"
server_key_version: "encryptionkey@0"
I20260924 21:12:39.723743 13050 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:12:39.724486 13065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:39.724763 13050 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:12:39.724869 13050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "34863bba7e8146cab737678d7499c335"
format_stamp: "Formatted at 2026-09-24 21:12:39 on dist-test-slave-q276"
server_key: "5a58c1ee2441304b88e35d6a64c8e5c2"
server_key_iv: "407da9cc88db3b89a8dc67a8df1ad32b"
server_key_version: "encryptionkey@0"
I20260924 21:12:39.724939 13050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:39.736608 13050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:39.738919 13050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:39.739077 13050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:39.743103 13050 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:39481
I20260924 21:12:39.743155 13117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:39481 every 8 connection(s)
I20260924 21:12:39.743497 13050 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:12:39.744545 13118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:39.746867 13118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335: Bootstrap starting.
I20260924 21:12:39.747416 13118 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:39.747655 13118 log.cc:824] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:39.748476 13118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335: No bootstrap required, opened a new log
I20260924 21:12:39.748816 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 13050
I20260924 21:12:39.748922 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:12:39.749107 6986 external_mini_cluster.cc:1428] Setting key 7072ebc40e6b1a61a2c977404ee2cfe8
I20260924 21:12:39.750690 13118 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } }
I20260924 21:12:39.750854 13118 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:39.750890 13118 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34863bba7e8146cab737678d7499c335, State: Initialized, Role: FOLLOWER
I20260924 21:12:39.751070 13118 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } }
I20260924 21:12:39.751168 13118 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:12:39.751202 13118 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:12:39.751256 13118 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:39.752188 13118 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } }
I20260924 21:12:39.752274 13118 leader_election.cc:302] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34863bba7e8146cab737678d7499c335; no voters:
I20260924 21:12:39.752442 13118 leader_election.cc:288] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:12:39.752485 13123 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:39.752602 13123 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [term 1 LEADER]: Becoming Leader. State: Replica: 34863bba7e8146cab737678d7499c335, State: Running, Role: LEADER
I20260924 21:12:39.752700 13118 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:12:39.752758 13123 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } }
I20260924 21:12:39.753093 13123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34863bba7e8146cab737678d7499c335. Latest consensus state: current_term: 1 leader_uuid: "34863bba7e8146cab737678d7499c335" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } }
I20260924 21:12:39.753227 13123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:39.753656 13131 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:12:39.753998 13124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34863bba7e8146cab737678d7499c335" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } }
I20260924 21:12:39.754160 13124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:39.754173 13131 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:12:39.755790 13131 catalog_manager.cc:1347] Generated new cluster ID: 91a4d8d24fc44b61b0e4d153ce600838
I20260924 21:12:39.755842 13131 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 21:12:39 dist-test-slave-q276 krb5kdc[13037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM
I20260924 21:12:39.761045 13122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:39.749611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56964 (local address 127.6.210.190:39481)
0924 21:12:39.749817 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:39.749825 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:39.749849 (+ 24us) server_negotiation.cc:407] Connection header received
0924 21:12:39.750677 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:39.750684 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:39.750732 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:39.750845 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:39.751622 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:39.752681 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:39.753757 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:39.754380 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:39.754483 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:39.757379 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:39.757383 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:39.757384 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:39.757396 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:39.759404 (+ 2008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:39.759791 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:39.759794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:39.759794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:39.759931 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:39.760145 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:39.760146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:39.760146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:39.760462 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:39.760628 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:39.760706 (+ 78us) server_negotiation.cc:299] Negotiation successful
0924 21:12:39.760845 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":119,"threads_started":1}
I20260924 21:12:39.768273 13131 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:12:39.768980 13131 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:12:39.772027 13131 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335: Generated new TSK 0
I20260924 21:12:39.772195 13131 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.6.210.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.210.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.210.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.129.keytab.
Entry for principal oryx/127.6.210.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.129.keytab.
I20260924 21:12:39.801164 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.129:0
--local_ip_for_outbound_sockets=127.6.210.129
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:39481
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.129.keytab
--principal=oryx/127.6.210.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:39813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:39.881949 13144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:39.882184 13144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:39.882222 13144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:39.883394 13144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:39.883462 13144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:39.883508 13144 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.129
I20260924 21:12:39.884999 13144 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:39813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.129.keytab
--principal=oryx/127.6.210.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:39481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13144
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.129
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:39.885273 13144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:39.885504 13144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:39.890618 13149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:39.890630 13150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:39.890810 13144 server_base.cc:1034] running on GCE node
W20260924 21:12:39.890846 13152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:39.891037 13144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:39.891288 13144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:39.892452 13144 hybrid_clock.cc:648] HybridClock initialized: now 1790284359892450 us; error 38 us; skew 500 ppm
Sep 24 21:12:39 dist-test-slave-q276 krb5kdc[13037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284359, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:39.895031 13144 init.cc:377] Logged in from keytab as oryx/127.6.210.129@KRBTEST.COM (short username oryx)
I20260924 21:12:39.895468 13144 webserver.cc:460] Webserver started at http://127.6.210.129:41683/ using document root <none> and password file <none>
I20260924 21:12:39.895709 13144 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:39.895777 13144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:39.895926 13144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:39.896984 13144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-0/data/instance:
uuid: "cb42303b63fc4ae2a46db49707af5edb"
format_stamp: "Formatted at 2026-09-24 21:12:39 on dist-test-slave-q276"
server_key: "efd49ef44cfad244533e46ab99e448ac"
server_key_iv: "f43947661c5201095c3bf7e087243625"
server_key_version: "encryptionkey@0"
I20260924 21:12:39.897359 13144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance:
uuid: "cb42303b63fc4ae2a46db49707af5edb"
format_stamp: "Formatted at 2026-09-24 21:12:39 on dist-test-slave-q276"
server_key: "efd49ef44cfad244533e46ab99e448ac"
server_key_iv: "f43947661c5201095c3bf7e087243625"
server_key_version: "encryptionkey@0"
I20260924 21:12:39.898761 13144 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:12:39.899770 13159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:39.899941 13144 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:12:39.900036 13144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-0/wal
uuid: "cb42303b63fc4ae2a46db49707af5edb"
format_stamp: "Formatted at 2026-09-24 21:12:39 on dist-test-slave-q276"
server_key: "efd49ef44cfad244533e46ab99e448ac"
server_key_iv: "f43947661c5201095c3bf7e087243625"
server_key_version: "encryptionkey@0"
I20260924 21:12:39.900125 13144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:39.926566 13144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:39.929255 13144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:39.929412 13144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:39.929653 13144 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:39.930052 13144 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:39.930110 13144 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:39.930166 13144 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:39.930218 13144 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:39.936312 13144 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.129:43821
I20260924 21:12:39.936365 13272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.129:43821 every 8 connection(s)
I20260924 21:12:39.936702 13144 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
Sep 24 21:12:39 dist-test-slave-q276 krb5kdc[13037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284359, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM
I20260924 21:12:39.946244 13122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:39.937547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:56965 (local address 127.6.210.190:39481)
0924 21:12:39.937607 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:39.937609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:39.937825 (+ 216us) server_negotiation.cc:407] Connection header received
0924 21:12:39.938423 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:39.938424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:39.938433 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:39.938496 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:39.939478 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:39.939967 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:39.941035 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:39.941239 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:39.941294 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:39.943569 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:39.943571 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:39.943571 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:39.943580 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:39.944882 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:39.945315 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:39.945316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:39.945316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:39.945372 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:39.945614 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:39.945614 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:39.945615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:39.945724 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:39.945801 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:39.946058 (+ 257us) server_negotiation.cc:299] Negotiation successful
0924 21:12:39.946155 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 21:12:39.946497 13275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:39.937513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.6.210.129:56965)
0924 21:12:39.937694 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:39.937739 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:39.938318 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:39.938566 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:39.938570 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:39.939187 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:39.939390 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:39.939395 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:39.940066 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:39.940067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:39.940953 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:39.940955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:39.941316 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:39.941317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:39.941427 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:39.942038 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:39.942048 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:39.943471 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:39.944979 (+ 1508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:39.944985 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:39.944987 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:39.945222 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:39.945438 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:39.945439 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:39.945439 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:39.945542 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:39.945792 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:39.945798 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:39.945932 (+ 134us) client_negotiation.cc:769] Sending connection context
0924 21:12:39.946038 (+ 106us) client_negotiation.cc:240] Negotiation successful
0924 21:12:39.946186 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":109,"threads_started":1}
I20260924 21:12:39.946630 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 13144
I20260924 21:12:39.946749 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance
I20260924 21:12:39.946969 6986 external_mini_cluster.cc:1428] Setting key c5feb4de66d0f86e79146c81b3ce6286
I20260924 21:12:39.946979 13273 heartbeater.cc:347] Connected to a master server at 127.6.210.190:39481
I20260924 21:12:39.947080 13273 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:39.947283 13273 heartbeater.cc:511] Master 127.6.210.190:39481 requested a full tablet report, sending...
I20260924 21:12:39.947781 13082 ts_manager.cc:194] Registered new tserver with Master: cb42303b63fc4ae2a46db49707af5edb (127.6.210.129:43821)
I20260924 21:12:39.948316 13082 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.210.129@KRBTEST.COM'} at 127.6.210.129:56965
WARNING: no policy specified for oryx/127.6.210.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.210.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.210.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.130.keytab.
Entry for principal oryx/127.6.210.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.130.keytab.
I20260924 21:12:39.971093 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.130:0
--local_ip_for_outbound_sockets=127.6.210.130
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:39481
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.130.keytab
--principal=oryx/127.6.210.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:39813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:40.055101 13278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:40.055320 13278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:40.055346 13278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:40.056574 13278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:40.056617 13278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:40.056648 13278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.130
I20260924 21:12:40.058197 13278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:39813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.130.keytab
--principal=oryx/127.6.210.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:39481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.130
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:40.058501 13278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:40.058717 13278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:40.063776 13286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:40.063781 13283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:40.063966 13278 server_base.cc:1034] running on GCE node
W20260924 21:12:40.063781 13284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:40.064270 13278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:40.064510 13278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:40.065661 13278 hybrid_clock.cc:648] HybridClock initialized: now 1790284360065648 us; error 27 us; skew 500 ppm
Sep 24 21:12:40 dist-test-slave-q276 krb5kdc[13037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284360, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:40.068177 13278 init.cc:377] Logged in from keytab as oryx/127.6.210.130@KRBTEST.COM (short username oryx)
I20260924 21:12:40.068552 13278 webserver.cc:460] Webserver started at http://127.6.210.130:45343/ using document root <none> and password file <none>
I20260924 21:12:40.068738 13278 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:40.068779 13278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:40.068864 13278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:40.069844 13278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-1/data/instance:
uuid: "d0982fc9b1f040af82515f52e5400c5f"
format_stamp: "Formatted at 2026-09-24 21:12:40 on dist-test-slave-q276"
server_key: "32a2c1e23f5b5aa4334cd9f50101804b"
server_key_iv: "2df42268f4c6077bf3160dbb04f9f22c"
server_key_version: "encryptionkey@0"
I20260924 21:12:40.070156 13278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance:
uuid: "d0982fc9b1f040af82515f52e5400c5f"
format_stamp: "Formatted at 2026-09-24 21:12:40 on dist-test-slave-q276"
server_key: "32a2c1e23f5b5aa4334cd9f50101804b"
server_key_iv: "2df42268f4c6077bf3160dbb04f9f22c"
server_key_version: "encryptionkey@0"
I20260924 21:12:40.071448 13278 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:40.072116 13293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:40.072291 13278 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:12:40.072391 13278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-1/wal
uuid: "d0982fc9b1f040af82515f52e5400c5f"
format_stamp: "Formatted at 2026-09-24 21:12:40 on dist-test-slave-q276"
server_key: "32a2c1e23f5b5aa4334cd9f50101804b"
server_key_iv: "2df42268f4c6077bf3160dbb04f9f22c"
server_key_version: "encryptionkey@0"
I20260924 21:12:40.072463 13278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:40.090663 13278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:40.092641 13278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:40.092813 13278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:40.093058 13278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:40.093379 13278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:40.093437 13278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:40.093477 13278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:40.093500 13278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:40.099298 13278 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.130:34739
I20260924 21:12:40.099346 13406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.130:34739 every 8 connection(s)
I20260924 21:12:40.099694 13278 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
I20260924 21:12:40.105772 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 13278
I20260924 21:12:40.105903 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance
Sep 24 21:12:40 dist-test-slave-q276 krb5kdc[13037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284360, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM
I20260924 21:12:40.106104 6986 external_mini_cluster.cc:1428] Setting key 1888ebc81571708e1966f3df2b2baa61
I20260924 21:12:40.109833 13122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.100451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:43025 (local address 127.6.210.190:39481)
0924 21:12:40.100523 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:40.100525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:40.100703 (+ 178us) server_negotiation.cc:407] Connection header received
0924 21:12:40.101315 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:40.101317 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:40.101328 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:40.101395 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:40.102377 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.102988 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.104062 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.104266 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.104329 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.106591 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:40.106592 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:40.106592 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:40.106600 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:40.108322 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.108787 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.108788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.108788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.108863 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.109188 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.109189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.109189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.109308 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:40.109364 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:40.109703 (+ 339us) server_negotiation.cc:299] Negotiation successful
0924 21:12:40.109768 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260924 21:12:40.110137 13409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.100423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.6.210.130:43025)
0924 21:12:40.100572 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:40.100613 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:40.101192 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:40.101455 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:40.101459 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:40.102024 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:40.102226 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.102230 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.103074 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.103075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.103901 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.103903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.104343 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.104344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.104452 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.104923 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:40.104931 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:40.106481 (+ 1550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:40.108419 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.108425 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.108427 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.108667 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.108937 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.108938 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.108938 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.109088 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.109373 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:40.109379 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:40.109556 (+ 177us) client_negotiation.cc:769] Sending connection context
0924 21:12:40.109673 (+ 117us) client_negotiation.cc:240] Negotiation successful
0924 21:12:40.109864 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":81,"threads_started":1}
I20260924 21:12:40.110723 13407 heartbeater.cc:347] Connected to a master server at 127.6.210.190:39481
I20260924 21:12:40.110834 13407 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:40.111008 13407 heartbeater.cc:511] Master 127.6.210.190:39481 requested a full tablet report, sending...
I20260924 21:12:40.111388 13082 ts_manager.cc:194] Registered new tserver with Master: d0982fc9b1f040af82515f52e5400c5f (127.6.210.130:34739)
I20260924 21:12:40.111944 13082 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.210.130@KRBTEST.COM'} at 127.6.210.130:43025
WARNING: no policy specified for oryx/127.6.210.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.210.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.210.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.131.keytab.
Entry for principal oryx/127.6.210.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.131.keytab.
I20260924 21:12:40.131376 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.131:0
--local_ip_for_outbound_sockets=127.6.210.131
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:39481
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.131.keytab
--principal=oryx/127.6.210.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:39813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:40.212118 13412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:40.212345 13412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:40.212385 13412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:40.213537 13412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:40.213609 13412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:40.213656 13412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.131
I20260924 21:12:40.215250 13412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:39813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.131.keytab
--principal=oryx/127.6.210.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:39481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.131
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:40.215518 13412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:40.215741 13412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:40.220796 13417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:40.220935 13412 server_base.cc:1034] running on GCE node
W20260924 21:12:40.220878 13418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:40.220798 13420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:40.221266 13412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:40.221532 13412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:40.222690 13412 hybrid_clock.cc:648] HybridClock initialized: now 1790284360222684 us; error 39 us; skew 500 ppm
Sep 24 21:12:40 dist-test-slave-q276 krb5kdc[13037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284360, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:40.225163 13412 init.cc:377] Logged in from keytab as oryx/127.6.210.131@KRBTEST.COM (short username oryx)
I20260924 21:12:40.225646 13412 webserver.cc:460] Webserver started at http://127.6.210.131:45533/ using document root <none> and password file <none>
I20260924 21:12:40.225870 13412 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:40.225939 13412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:40.226086 13412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:40.227290 13412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-2/data/instance:
uuid: "5ecda7f62fe647d78e4a990968bf5327"
format_stamp: "Formatted at 2026-09-24 21:12:40 on dist-test-slave-q276"
server_key: "351779a6770d3cfa2cb6ae624833bd81"
server_key_iv: "e2d35f62712e7e24298495636e1bd353"
server_key_version: "encryptionkey@0"
I20260924 21:12:40.227656 13412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance:
uuid: "5ecda7f62fe647d78e4a990968bf5327"
format_stamp: "Formatted at 2026-09-24 21:12:40 on dist-test-slave-q276"
server_key: "351779a6770d3cfa2cb6ae624833bd81"
server_key_iv: "e2d35f62712e7e24298495636e1bd353"
server_key_version: "encryptionkey@0"
I20260924 21:12:40.228988 13412 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:12:40.229717 13427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:40.229883 13412 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:12:40.229986 13412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-2/wal
uuid: "5ecda7f62fe647d78e4a990968bf5327"
format_stamp: "Formatted at 2026-09-24 21:12:40 on dist-test-slave-q276"
server_key: "351779a6770d3cfa2cb6ae624833bd81"
server_key_iv: "e2d35f62712e7e24298495636e1bd353"
server_key_version: "encryptionkey@0"
I20260924 21:12:40.230057 13412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:40.237159 13412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:40.238986 13412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:40.239147 13412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:40.239359 13412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:40.239660 13412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:40.239719 13412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:40.239758 13412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:40.239786 13412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:40.245425 13412 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.131:37515
I20260924 21:12:40.245465 13540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.131:37515 every 8 connection(s)
I20260924 21:12:40.245829 13412 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
Sep 24 21:12:40 dist-test-slave-q276 krb5kdc[13037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284360, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM
I20260924 21:12:40.255201 13122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.246552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:35139 (local address 127.6.210.190:39481)
0924 21:12:40.246618 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:40.246621 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:40.246807 (+ 186us) server_negotiation.cc:407] Connection header received
0924 21:12:40.247324 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:40.247325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:40.247338 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:40.247395 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:40.248339 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.248839 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.249861 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.250056 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.250202 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.252289 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:40.252290 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:40.252291 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:40.252298 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:40.253595 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.254087 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.254087 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.254088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.254181 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.254525 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.254526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.254527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.254656 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:40.254736 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:40.255017 (+ 281us) server_negotiation.cc:299] Negotiation successful
0924 21:12:40.255101 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 21:12:40.255383 13543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.246530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.6.210.131:35139)
0924 21:12:40.246678 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:40.246724 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:40.247216 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:40.247467 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:40.247472 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:40.248040 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:40.248228 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.248233 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.248932 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.248933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.249752 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.249753 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.250211 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.250212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.250385 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.250853 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:40.250860 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:40.252193 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:40.253729 (+ 1536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.253736 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.253738 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.254024 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.254306 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.254307 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.254307 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.254427 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.254726 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:40.254732 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:40.254872 (+ 140us) client_negotiation.cc:769] Sending connection context
0924 21:12:40.254983 (+ 111us) client_negotiation.cc:240] Negotiation successful
0924 21:12:40.255144 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":74,"threads_started":1}
I20260924 21:12:40.255873 13541 heartbeater.cc:347] Connected to a master server at 127.6.210.190:39481
I20260924 21:12:40.255929 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 13412
I20260924 21:12:40.255968 13541 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:40.256050 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance
I20260924 21:12:40.256184 13541 heartbeater.cc:511] Master 127.6.210.190:39481 requested a full tablet report, sending...
I20260924 21:12:40.256275 6986 external_mini_cluster.cc:1428] Setting key 1f3d538c5d2716d0069c8448621997ab
I20260924 21:12:40.256600 13082 ts_manager.cc:194] Registered new tserver with Master: 5ecda7f62fe647d78e4a990968bf5327 (127.6.210.131:37515)
I20260924 21:12:40.257130 6986 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:12:40.257130 13082 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.210.131@KRBTEST.COM'} at 127.6.210.131:35139
WARNING: no policy specified for oryx/127.6.210.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.210.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.210.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.189.keytab.
Entry for principal oryx/127.6.210.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.189.keytab.
I20260924 21:12:40.283262 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.189:44341
--webserver_interface=127.6.210.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.189.keytab
--principal=oryx/127.6.210.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:39813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:39481,127.6.210.189:44341
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:40.364039 13546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:40.364240 13546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:40.364262 13546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:40.365358 13546 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:12:40.365393 13546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:40.365407 13546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:40.365419 13546 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:12:40.365432 13546 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:12:40.366820 13546 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:39813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:39481,127.6.210.189:44341
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.189.keytab
--principal=oryx/127.6.210.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.189:44341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.210.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
W20260924 21:12:40.366922 13546 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.6.210.190:39481,127.6.210.189:44341'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260924 21:12:40.367027 13546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:40.367184 13546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:40.371984 13551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:40.371996 13554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:40.372112 13552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:40.372262 13546 server_base.cc:1034] running on GCE node
I20260924 21:12:40.372473 13546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:40.372758 13546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:40.373916 13546 hybrid_clock.cc:648] HybridClock initialized: now 1790284360373906 us; error 26 us; skew 500 ppm
Sep 24 21:12:40 dist-test-slave-q276 krb5kdc[13037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284360, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:40.376426 13546 init.cc:377] Logged in from keytab as oryx/127.6.210.189@KRBTEST.COM (short username oryx)
I20260924 21:12:40.376909 13546 webserver.cc:460] Webserver started at http://127.6.210.189:36975/ using document root <none> and password file <none>
I20260924 21:12:40.377132 13546 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:40.377177 13546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:40.377312 13546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:40.378439 13546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-1/data/instance:
uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776"
format_stamp: "Formatted at 2026-09-24 21:12:40 on dist-test-slave-q276"
server_key: "f3e312ce2d3f99b7f9a3c54c8004e3b3"
server_key_iv: "de80cb0f8228db07f92bd0583d1055d4"
server_key_version: "encryptionkey@0"
I20260924 21:12:40.378780 13546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-1/wal/instance:
uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776"
format_stamp: "Formatted at 2026-09-24 21:12:40 on dist-test-slave-q276"
server_key: "f3e312ce2d3f99b7f9a3c54c8004e3b3"
server_key_iv: "de80cb0f8228db07f92bd0583d1055d4"
server_key_version: "encryptionkey@0"
I20260924 21:12:40.380069 13546 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:40.380774 13561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:40.380956 13546 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:12:40.381042 13546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-1/wal
uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776"
format_stamp: "Formatted at 2026-09-24 21:12:40 on dist-test-slave-q276"
server_key: "f3e312ce2d3f99b7f9a3c54c8004e3b3"
server_key_iv: "de80cb0f8228db07f92bd0583d1055d4"
server_key_version: "encryptionkey@0"
I20260924 21:12:40.381114 13546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:40.388509 13546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:40.390175 13546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:40.390347 13546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:40.394018 13546 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.189:44341
I20260924 21:12:40.394068 13613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.189:44341 every 8 connection(s)
I20260924 21:12:40.394399 13546 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-1/data/info.pb
I20260924 21:12:40.395299 13614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:40.396726 13614 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } has no permanent_uuid. Determining permanent_uuid...
I20260924 21:12:40.397797 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 13546
Sep 24 21:12:40 dist-test-slave-q276 krb5kdc[13037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284360, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.189@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM
Sep 24 21:12:40 dist-test-slave-q276 krb5kdc[13037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.210.189@KRBTEST.COM
I20260924 21:12:40.407204 13122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.397348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56978 (local address 127.6.210.190:39481)
0924 21:12:40.397423 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:40.397426 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:40.397582 (+ 156us) server_negotiation.cc:407] Connection header received
0924 21:12:40.398070 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:40.398071 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:40.398083 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:40.398150 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:40.398945 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.399580 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.400854 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.401143 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.401220 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.403869 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:40.403870 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:40.403871 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:40.403878 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:40.405480 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.405971 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.405972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.405972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.406024 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.406348 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.406349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.406350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.406524 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:40.406605 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:40.406930 (+ 325us) server_negotiation.cc:299] Negotiation successful
0924 21:12:40.407021 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260924 21:12:40.407240 13616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.397293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.0.0.1:56978)
0924 21:12:40.397467 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:40.397500 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:40.397963 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:40.398225 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:40.398230 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:40.398638 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:40.398861 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.398866 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.399688 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.399689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.400781 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.400783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.401247 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.401248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.401378 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.402046 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:40.402053 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:40.403792 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:40.405559 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.405566 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.405569 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.405880 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.406100 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.406101 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.406102 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.406235 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.406606 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:40.406612 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:40.406772 (+ 160us) client_negotiation.cc:769] Sending connection context
0924 21:12:40.406847 (+ 75us) client_negotiation.cc:240] Negotiation successful
0924 21:12:40.406980 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":174,"thread_start_us":90,"threads_started":1}
I20260924 21:12:40.408037 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.398317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58706 (local address 127.6.210.189:44341)
0924 21:12:40.398460 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:40.398464 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:40.398483 (+ 19us) server_negotiation.cc:407] Connection header received
0924 21:12:40.398542 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:40.398544 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:40.398566 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:40.398651 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:40.399305 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.400394 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.401245 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.401700 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.401770 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.404861 (+ 3091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:40.404867 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:40.404867 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:40.404873 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:40.406605 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.407006 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.407010 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.407010 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.407062 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.407489 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.407490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.407490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.407629 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:40.407739 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:40.407883 (+ 144us) server_negotiation.cc:299] Negotiation successful
0924 21:12:40.407960 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":118,"thread_start_us":95,"threads_started":1}
I20260924 21:12:40.408015 13614 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 21:12:40 dist-test-slave-q276 krb5kdc[13037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284360, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.189@KRBTEST.COM for oryx/127.6.210.189@KRBTEST.COM
I20260924 21:12:40.414158 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.408380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58708 (local address 127.6.210.189:44341)
0924 21:12:40.408441 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:40.408442 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:40.408454 (+ 12us) server_negotiation.cc:407] Connection header received
0924 21:12:40.408467 (+ 13us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:40.408467 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:40.408473 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:40.408524 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:40.408802 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.409260 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.409905 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.410104 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.410193 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.411886 (+ 1693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:40.411888 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:40.411888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:40.411895 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:40.413379 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.413622 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.413623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.413623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.413664 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.413795 (+ 131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.413795 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.413796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.413888 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:40.413935 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:40.414031 (+ 96us) server_negotiation.cc:299] Negotiation successful
0924 21:12:40.414099 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260924 21:12:40.414331 13616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.408309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:44341 (local address 127.0.0.1:58708)
0924 21:12:40.408339 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:40.408350 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:40.408386 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:40.408539 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:40.408540 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:40.408694 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:40.408732 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.408734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.409348 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.409349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.409817 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.409819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.410188 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.410189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.410241 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.410608 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:40.410612 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:40.411808 (+ 1196us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:40.413446 (+ 1638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.413446 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.413447 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.413579 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.413714 (+ 135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.413714 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.413714 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.413757 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.413937 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:40.413937 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:40.413968 (+ 31us) client_negotiation.cc:769] Sending connection context
0924 21:12:40.414014 (+ 46us) client_negotiation.cc:240] Negotiation successful
0924 21:12:40.414157 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16,"mutex_wait_us":35}
I20260924 21:12:40.415676 13614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776: Bootstrap starting.
I20260924 21:12:40.416486 13614 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:40.416760 13614 log.cc:824] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:40.417474 13614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776: No bootstrap required, opened a new log
I20260924 21:12:40.419122 13614 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } }
I20260924 21:12:40.419253 13614 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:40.419277 13614 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bbea6a1f4d844e9a0dedfbaa1c60776, State: Initialized, Role: FOLLOWER
I20260924 21:12:40.419370 13614 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } }
I20260924 21:12:40.419622 13620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } } }
I20260924 21:12:40.419692 13614 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:12:40.419739 13620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [sys.catalog]: This master's current role is: FOLLOWER
W20260924 21:12:40.421298 13631 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 21:12:40.421370 13631 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 21:12:40.422986 13546 master_runner.cc:428] Detected that this master 9bbea6a1f4d844e9a0dedfbaa1c60776 is joining an existing cluster
I20260924 21:12:40.423038 13546 master_runner.cc:432] Initiating AddMaster RPC to add 127.6.210.189:44341
I20260924 21:12:40.431203 13638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.424114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:44341 (local address 127.0.0.1:58716)
0924 21:12:40.424349 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:40.424357 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:40.424419 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:40.424576 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:40.424577 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:40.424827 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:40.424911 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.424913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.425944 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.425945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.426572 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.426573 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.426909 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.426910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.426972 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.427442 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:40.427447 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:40.427835 (+ 388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:40.429940 (+ 2105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.429940 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.429941 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.430163 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.430393 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.430394 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.430394 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.430448 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.430760 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:40.430761 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:40.430814 (+ 53us) client_negotiation.cc:769] Sending connection context
0924 21:12:40.430916 (+ 102us) client_negotiation.cc:240] Negotiation successful
0924 21:12:40.431041 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":3,"thread_start_us":99,"threads_started":1}
I20260924 21:12:40.431282 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.424293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58716 (local address 127.6.210.189:44341)
0924 21:12:40.424334 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:40.424336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:40.424401 (+ 65us) server_negotiation.cc:407] Connection header received
0924 21:12:40.424488 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:40.424488 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:40.424496 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:40.424541 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:40.425001 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.425846 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.426650 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.426829 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.426910 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.427924 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:40.427925 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:40.427926 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:40.427932 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:40.429849 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.430254 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.430255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.430256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.430325 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.430528 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.430529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.430529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.430668 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:40.430749 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:40.430949 (+ 200us) server_negotiation.cc:299] Negotiation successful
0924 21:12:40.431117 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":90,"server-negotiator.queue_time_us":22}
I20260924 21:12:40.432844 13122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.424216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56990 (local address 127.6.210.190:39481)
0924 21:12:40.424266 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:40.424269 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:40.424379 (+ 110us) server_negotiation.cc:407] Connection header received
0924 21:12:40.424447 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:40.424448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:40.424460 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:40.424541 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:40.425957 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.426823 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.428038 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.428305 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.428390 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.429515 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:40.429516 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:40.429517 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:40.429524 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:40.431295 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.432041 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.432042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.432043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.432096 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.432304 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.432304 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.432305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.432415 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:40.432490 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:40.432676 (+ 186us) server_negotiation.cc:299] Negotiation successful
0924 21:12:40.432754 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 21:12:40.432873 13637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.424114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.0.0.1:56990)
0924 21:12:40.424314 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:40.424327 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:40.424392 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:40.424543 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:40.424544 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:40.424825 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:40.424911 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.424913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.426930 (+ 2017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.426931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.427926 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.427928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.428440 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.428441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.428537 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.428958 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:40.428962 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:40.429415 (+ 453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:40.431813 (+ 2398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.431814 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.431814 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.431963 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.432178 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.432179 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.432179 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.432233 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.432481 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:40.432482 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:40.432534 (+ 52us) client_negotiation.cc:769] Sending connection context
0924 21:12:40.432620 (+ 86us) client_negotiation.cc:240] Negotiation successful
0924 21:12:40.432772 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":2,"spinlock_wait_cycles":10112,"thread_start_us":101,"threads_started":1}
Sep 24 21:12:40 dist-test-slave-q276 krb5kdc[13037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284359, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.190@KRBTEST.COM for oryx/127.6.210.189@KRBTEST.COM
I20260924 21:12:40.446022 13618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.435235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58726 (local address 127.6.210.189:44341)
0924 21:12:40.435268 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:40.435269 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:40.438518 (+ 3249us) server_negotiation.cc:407] Connection header received
0924 21:12:40.438621 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:40.438622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:40.438630 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:40.438681 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:40.439195 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.439626 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.440860 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.441030 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.441118 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.443413 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:40.443415 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:40.443415 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:40.443422 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:40.444793 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.445211 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.445212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.445213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.445273 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.445470 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.445471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.445472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.445585 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:40.445657 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:40.445845 (+ 188us) server_negotiation.cc:299] Negotiation successful
0924 21:12:40.445921 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 21:12:40.446053 13639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.435180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:44341 (local address 127.0.0.1:58726)
0924 21:12:40.438393 (+ 3213us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:40.438448 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:40.438533 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:40.438686 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:40.438689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:40.439055 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:40.439119 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.439121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.439702 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.439703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.440766 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.440770 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.441126 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.441128 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.441233 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.441703 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:40.441709 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:40.443306 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:40.444894 (+ 1588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.444903 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.444906 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.445116 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.445340 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.445341 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.445341 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.445408 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.445659 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:40.445667 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:40.445721 (+ 54us) client_negotiation.cc:769] Sending connection context
0924 21:12:40.445821 (+ 100us) client_negotiation.cc:240] Negotiation successful
0924 21:12:40.445942 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3179,"thread_start_us":78,"threads_started":1}
I20260924 21:12:40.446617 13082 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.6.210.189:44341: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776"
member_type: NON_VOTER
last_known_addr {
host: "127.6.210.189"
port: 44341
}
attrs {
promote: true
}
}
dest_uuid: "34863bba7e8146cab737678d7499c335"
cas_config_opid_index: -1
I20260924 21:12:40.446959 13082 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: NON_VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: true } }
I20260924 21:12:40.447469 13641 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 9bbea6a1f4d844e9a0dedfbaa1c60776 (127.6.210.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: NON_VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: true } } }
I20260924 21:12:40.447728 13588 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:40.448601 13641 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.6.210.189:44341
I20260924 21:12:40.448621 13588 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [term 1 FOLLOWER]: Refusing update from remote peer 34863bba7e8146cab737678d7499c335: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260924 21:12:40.448850 13546 master.cc:537] Master@127.6.210.189:44341 shutting down...
I20260924 21:12:40.448848 13641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "34863bba7e8146cab737678d7499c335" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: NON_VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: true } } }
I20260924 21:12:40.448920 13641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:40.448899 13640 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: NON_VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 21:12:40.449543 13067 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260924 21:12:40.449949 13067 consensus_peers.cc:487] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 -> Peer 9bbea6a1f4d844e9a0dedfbaa1c60776 (127.6.210.189:44341): Couldn't send request to peer 9bbea6a1f4d844e9a0dedfbaa1c60776. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260924 21:12:40.452126 13546 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:12:40.452252 13546 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:40.452304 13546 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776: stopping tablet replica
I20260924 21:12:40.464731 13546 master.cc:552] Master@127.6.210.189:44341 shutdown complete.
I20260924 21:12:40.464782 13546 master_runner.cc:305] Clearing existing system tablet
I20260924 21:12:40.464927 13546 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:40.464977 13546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:40.465598 13546 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:12:40.466101 13646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:40.466233 13546 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:40.466302 13546 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:12:40.466327 13546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-1/wal
uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776"
format_stamp: "Formatted at 2026-09-24 21:12:40 on dist-test-slave-q276"
server_key: "f3e312ce2d3f99b7f9a3c54c8004e3b3"
server_key_iv: "de80cb0f8228db07f92bd0583d1055d4"
server_key_version: "encryptionkey@0"
I20260924 21:12:40.467351 13546 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:40.468699 13546 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260924 21:12:40.468757 13546 log.cc:1196] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260924 21:12:40.468954 13546 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776: Deleting consensus metadata
I20260924 21:12:40.469076 13546 master_runner.cc:315] Copying system tablet from 127.6.210.190:39481
I20260924 21:12:40.469720 13546 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776: tablet copy: Beginning tablet copy session from remote peer at address 127.6.210.190:39481
I20260924 21:12:40.476991 13122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.470120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56994 (local address 127.6.210.190:39481)
0924 21:12:40.470166 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:40.470176 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:40.470219 (+ 43us) server_negotiation.cc:407] Connection header received
0924 21:12:40.470337 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:40.470338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:40.470348 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:40.470407 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:40.470858 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.471439 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.472698 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.472931 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.472998 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.474118 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:40.474120 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:40.474120 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:40.474126 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:40.475833 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.476249 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.476250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.476250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.476308 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.476512 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.476512 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.476513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.476627 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:40.476684 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:40.476863 (+ 179us) server_negotiation.cc:299] Negotiation successful
0924 21:12:40.476924 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 21:12:40.477214 13652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.470017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.0.0.1:56994)
0924 21:12:40.470142 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:40.470154 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:40.470231 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:40.470431 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:40.470432 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:40.470670 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:40.470749 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.470752 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.471532 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.471533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.472599 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.472603 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.473012 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.473013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.473087 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.473574 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:40.473580 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:40.474010 (+ 430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:40.475910 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.475911 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.475912 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.476157 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.476372 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.476373 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.476373 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.476444 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.476707 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:40.476708 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:40.476760 (+ 52us) client_negotiation.cc:769] Sending connection context
0924 21:12:40.476846 (+ 86us) client_negotiation.cc:240] Negotiation successful
0924 21:12:40.476956 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":80,"threads_started":1}
I20260924 21:12:40.477617 13102 tablet_copy_service.cc:140] P 34863bba7e8146cab737678d7499c335: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 9bbea6a1f4d844e9a0dedfbaa1c60776 ({username='oryx', principal='oryx/127.6.210.189@KRBTEST.COM'} at 127.0.0.1:56994)
I20260924 21:12:40.477730 13102 tablet_copy_service.cc:161] P 34863bba7e8146cab737678d7499c335: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 9bbea6a1f4d844e9a0dedfbaa1c60776 at {username='oryx', principal='oryx/127.6.210.189@KRBTEST.COM'} at 127.0.0.1:56994: session id = 9bbea6a1f4d844e9a0dedfbaa1c60776-00000000000000000000000000000000
I20260924 21:12:40.478628 13102 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 21:12:40.479281 13546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260924 21:12:40.480254 13546 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776: tablet copy: Starting download of 0 data blocks...
I20260924 21:12:40.480422 13546 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776: tablet copy: Starting download of 1 WAL segments...
I20260924 21:12:40.481429 13546 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 21:12:40.482255 13102 tablet_copy_service.cc:342] P 34863bba7e8146cab737678d7499c335: Request end of tablet copy session 9bbea6a1f4d844e9a0dedfbaa1c60776-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.6.210.189@KRBTEST.COM'} at 127.0.0.1:56994
I20260924 21:12:40.482357 13102 tablet_copy_service.cc:433] P 34863bba7e8146cab737678d7499c335: ending tablet copy session 9bbea6a1f4d844e9a0dedfbaa1c60776-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 9bbea6a1f4d844e9a0dedfbaa1c60776
I20260924 21:12:40.485003 13546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:40.486351 13653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:40.486403 13656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:40.486410 13654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:40.486640 13546 server_base.cc:1034] running on GCE node
I20260924 21:12:40.486734 13546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:40.486876 13546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:40.488006 13546 hybrid_clock.cc:648] HybridClock initialized: now 1790284360488004 us; error 31 us; skew 500 ppm
Sep 24 21:12:40 dist-test-slave-q276 krb5kdc[13037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284360, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:40.489327 13546 init.cc:377] Logged in from keytab as oryx/127.6.210.189@KRBTEST.COM (short username oryx)
I20260924 21:12:40.489600 13546 webserver.cc:460] Webserver started at http://127.6.210.189:43261/ using document root <none> and password file <none>
I20260924 21:12:40.489725 13546 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:40.489782 13546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:40.490459 13546 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:12:40.491016 13663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:40.491207 13546 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:12:40.491266 13546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-1/wal
uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776"
format_stamp: "Formatted at 2026-09-24 21:12:40 on dist-test-slave-q276"
server_key: "f3e312ce2d3f99b7f9a3c54c8004e3b3"
server_key_iv: "de80cb0f8228db07f92bd0583d1055d4"
server_key_version: "encryptionkey@0"
I20260924 21:12:40.491326 13546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:40.502808 13546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:40.504666 13546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:40.507781 13546 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.189:44341
I20260924 21:12:40.508085 13715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.189:44341 every 8 connection(s)
I20260924 21:12:40.508183 13546 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-1/data/info.pb
I20260924 21:12:40.508865 13716 sys_catalog.cc:263] Verifying existing consensus state
I20260924 21:12:40.509428 13716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776: Bootstrap starting.
I20260924 21:12:40.512018 13716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 21:12:40.512279 13716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776: Bootstrap complete.
I20260924 21:12:40.512483 13716 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: NON_VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: true } }
I20260924 21:12:40.512575 13716 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9bbea6a1f4d844e9a0dedfbaa1c60776, State: Initialized, Role: LEARNER
I20260924 21:12:40.512620 13716 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: NON_VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: true } }
I20260924 21:12:40.512792 13716 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:12:40.512784 13720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: NON_VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: true } } }
I20260924 21:12:40.512984 13720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [sys.catalog]: This master's current role is: LEARNER
I20260924 21:12:40.514773 13731 catalog_manager.cc:1259] Loaded cluster ID: 91a4d8d24fc44b61b0e4d153ce600838
I20260924 21:12:40.514833 13731 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776: loading cluster ID for follower catalog manager: success
I20260924 21:12:40.516299 13731 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776: acquiring CA information for follower catalog manager: success
I20260924 21:12:40.516981 13731 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 21:12:40.518564 13718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.509664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58740 (local address 127.6.210.189:44341)
0924 21:12:40.510392 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:40.510394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:40.510406 (+ 12us) server_negotiation.cc:407] Connection header received
0924 21:12:40.510432 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:40.510433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:40.510444 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:40.510533 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:40.511202 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.512334 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.513489 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.513846 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.513928 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.515748 (+ 1820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:40.515750 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:40.515750 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:40.515757 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:40.517371 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.517738 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.517739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.517740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.517800 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.518007 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.518008 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.518008 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.518122 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:40.518230 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:40.518363 (+ 133us) server_negotiation.cc:299] Negotiation successful
0924 21:12:40.518450 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":700,"thread_start_us":74,"threads_started":1}
WARNING: no policy specified for oryx/127.6.210.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.210.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.210.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.188.keytab.
Entry for principal oryx/127.6.210.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.188.keytab.
I20260924 21:12:40.549867 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.188:46595
--webserver_interface=127.6.210.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.188.keytab
--principal=oryx/127.6.210.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:39813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:39481,127.6.210.189:44341,127.6.210.188:46595
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:40.636893 13734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:40.637108 13734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:40.637131 13734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:40.638346 13734 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:12:40.638391 13734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:40.638404 13734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:40.638417 13734 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:12:40.638428 13734 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:12:40.640002 13734 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:39813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:39481,127.6.210.189:44341,127.6.210.188:46595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.188.keytab
--principal=oryx/127.6.210.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.188:46595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.210.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:40.640269 13734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:40.640543 13734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:40.645478 13739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:40.645537 13740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:40.645692 13742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:40.645710 13734 server_base.cc:1034] running on GCE node
I20260924 21:12:40.645963 13734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:40.646344 13734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:40.647516 13734 hybrid_clock.cc:648] HybridClock initialized: now 1790284360647499 us; error 46 us; skew 500 ppm
Sep 24 21:12:40 dist-test-slave-q276 krb5kdc[13037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284360, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:40.650580 13734 init.cc:377] Logged in from keytab as oryx/127.6.210.188@KRBTEST.COM (short username oryx)
I20260924 21:12:40.651028 13734 webserver.cc:460] Webserver started at http://127.6.210.188:40289/ using document root <none> and password file <none>
I20260924 21:12:40.651268 13734 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:40.651314 13734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:40.651480 13734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:40.652542 13734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-2/data/instance:
uuid: "faabfe445f404446a8f8eef00b4b6037"
format_stamp: "Formatted at 2026-09-24 21:12:40 on dist-test-slave-q276"
server_key: "9e48d1d9e358962f65102bcca80948bc"
server_key_iv: "4f384b1fed6a538c9f160a8edaced363"
server_key_version: "encryptionkey@0"
I20260924 21:12:40.652997 13734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-2/wal/instance:
uuid: "faabfe445f404446a8f8eef00b4b6037"
format_stamp: "Formatted at 2026-09-24 21:12:40 on dist-test-slave-q276"
server_key: "9e48d1d9e358962f65102bcca80948bc"
server_key_iv: "4f384b1fed6a538c9f160a8edaced363"
server_key_version: "encryptionkey@0"
I20260924 21:12:40.654500 13734 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:12:40.655251 13749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:40.655441 13734 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:12:40.655535 13734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-2/wal
uuid: "faabfe445f404446a8f8eef00b4b6037"
format_stamp: "Formatted at 2026-09-24 21:12:40 on dist-test-slave-q276"
server_key: "9e48d1d9e358962f65102bcca80948bc"
server_key_iv: "4f384b1fed6a538c9f160a8edaced363"
server_key_version: "encryptionkey@0"
I20260924 21:12:40.655622 13734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:40.678160 13734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:40.680583 13734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:40.680732 13734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:40.684747 13734 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.188:46595
I20260924 21:12:40.684783 13801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.188:46595 every 8 connection(s)
I20260924 21:12:40.685142 13734 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-2/data/info.pb
I20260924 21:12:40.686097 13802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:40.688207 13802 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 21:12:40 dist-test-slave-q276 krb5kdc[13037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284360, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.188@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM
I20260924 21:12:40.695308 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 13734
I20260924 21:12:40.697376 13122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.689052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57004 (local address 127.6.210.190:39481)
0924 21:12:40.689128 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:40.689130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:40.689233 (+ 103us) server_negotiation.cc:407] Connection header received
0924 21:12:40.689689 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:40.689690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:40.689701 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:40.689767 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:40.690423 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.690906 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.692021 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.692234 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.692293 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.694412 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:40.694413 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:40.694414 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:40.694420 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:40.695922 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.696399 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.696401 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.696401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.696466 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.696722 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.696723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.696723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.696870 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:40.696934 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:40.697251 (+ 317us) server_negotiation.cc:299] Negotiation successful
0924 21:12:40.697314 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 21:12:40.697587 13804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.688900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.0.0.1:57004)
0924 21:12:40.689104 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:40.689148 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:40.689577 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:40.689837 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:40.689841 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:40.690078 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:40.690339 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.690344 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.691014 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.691015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.691935 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.691937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.692333 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.692334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.692502 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.692962 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:40.692969 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:40.694333 (+ 1364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:40.696012 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.696016 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.696018 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.696324 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.696545 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.696546 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.696546 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.696650 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.696933 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:40.696937 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:40.697080 (+ 143us) client_negotiation.cc:769] Sending connection context
0924 21:12:40.697222 (+ 142us) client_negotiation.cc:240] Negotiation successful
0924 21:12:40.697370 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":97,"threads_started":1}
I20260924 21:12:40.698098 13802 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 21:12:40 dist-test-slave-q276 krb5kdc[13037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.210.188@KRBTEST.COM
Sep 24 21:12:40 dist-test-slave-q276 krb5kdc[13037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284360, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.188@KRBTEST.COM for oryx/127.6.210.189@KRBTEST.COM
I20260924 21:12:40.705123 13805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.695836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40896 (local address 127.6.210.188:46595)
0924 21:12:40.696015 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:40.696020 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:40.696041 (+ 21us) server_negotiation.cc:407] Connection header received
0924 21:12:40.696061 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:40.696063 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:40.696075 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:40.696156 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:40.696840 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.697865 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.698781 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.699130 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.699184 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.701925 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:40.701934 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:40.701934 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:40.701941 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:40.703830 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.704252 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.704256 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.704256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.704319 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.704530 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.704531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.704531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.704668 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:40.704847 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:40.704914 (+ 67us) server_negotiation.cc:299] Negotiation successful
0924 21:12:40.704996 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":92,"threads_started":1}
I20260924 21:12:40.705322 13804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.698359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:44341 (local address 127.0.0.1:58752)
0924 21:12:40.698405 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:40.698415 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:40.698451 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:40.698579 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:40.698580 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:40.698710 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:40.698772 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.698774 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.699518 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.699519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.700134 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.700136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.700654 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.700655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.700702 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.701027 (+ 325us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:40.701032 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:40.702423 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:40.704098 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.704099 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.704099 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.704262 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.704465 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.704465 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.704466 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.704523 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.704782 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:40.704783 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:40.704823 (+ 40us) client_negotiation.cc:769] Sending connection context
0924 21:12:40.704905 (+ 82us) client_negotiation.cc:240] Negotiation successful
0924 21:12:40.705049 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":57}
I20260924 21:12:40.705214 13718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.698419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58752 (local address 127.6.210.189:44341)
0924 21:12:40.698469 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:40.698472 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:40.698481 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:12:40.698504 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:40.698505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:40.698514 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:40.698577 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:40.698863 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.699424 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.700223 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.700570 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.700644 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.702546 (+ 1902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:40.702576 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:40.702577 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:40.702585 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:40.703995 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.704337 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.704338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.704338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.704398 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.704586 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.704587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.704588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.704713 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:40.704790 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:40.704943 (+ 153us) server_negotiation.cc:299] Negotiation successful
0924 21:12:40.705034 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 21:12:40.705806 13802 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.210.188" port: 46595 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 21:12:40 dist-test-slave-q276 krb5kdc[13037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284360, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.188@KRBTEST.COM for oryx/127.6.210.188@KRBTEST.COM
I20260924 21:12:40.711980 13804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.706053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:46595 (local address 127.0.0.1:40912)
0924 21:12:40.706098 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:40.706109 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:40.706145 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:40.706248 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:40.706248 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:40.706408 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:40.706435 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.706436 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.707289 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.707290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.707900 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.707902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.708241 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.708242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.708284 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.708633 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:40.708637 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:40.709769 (+ 1132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:40.711166 (+ 1397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.711167 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.711167 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.711330 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.711496 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.711496 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.711496 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.711545 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.711725 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:40.711726 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:40.711759 (+ 33us) client_negotiation.cc:769] Sending connection context
0924 21:12:40.711817 (+ 58us) client_negotiation.cc:240] Negotiation successful
0924 21:12:40.711891 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260924 21:12:40.712026 13805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.706110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40912 (local address 127.6.210.188:46595)
0924 21:12:40.706140 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:40.706141 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:40.706147 (+ 6us) server_negotiation.cc:407] Connection header received
0924 21:12:40.706202 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:40.706202 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:40.706210 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:40.706246 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:40.706494 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.707159 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.707973 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.708159 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.708226 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.709850 (+ 1624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:40.709851 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:40.709852 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:40.709857 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:40.711080 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.711395 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.711395 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.711395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.711439 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.711593 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.711593 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.711593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.711671 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:40.711719 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:40.711834 (+ 115us) server_negotiation.cc:299] Negotiation successful
0924 21:12:40.711932 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":15}
I20260924 21:12:40.713397 13802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037: Bootstrap starting.
I20260924 21:12:40.714309 13802 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:40.714593 13802 log.cc:824] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:40.715322 13802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037: No bootstrap required, opened a new log
I20260924 21:12:40.716704 13802 raft_consensus.cc:348] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 46595 } }
I20260924 21:12:40.716821 13802 raft_consensus.cc:374] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:40.716842 13802 raft_consensus.cc:729] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faabfe445f404446a8f8eef00b4b6037, State: Initialized, Role: FOLLOWER
I20260924 21:12:40.716987 13802 consensus_queue.cc:260] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 46595 } }
I20260924 21:12:40.717240 13807 sys_catalog.cc:455] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 46595 } } }
I20260924 21:12:40.717298 13802 sys_catalog.cc:564] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:12:40.717360 13807 sys_catalog.cc:458] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [sys.catalog]: This master's current role is: FOLLOWER
W20260924 21:12:40.718957 13818 catalog_manager.cc:1558] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 21:12:40.719018 13818 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 21:12:40.721483 13734 master_runner.cc:428] Detected that this master faabfe445f404446a8f8eef00b4b6037 is joining an existing cluster
I20260924 21:12:40.721539 13734 master_runner.cc:432] Initiating AddMaster RPC to add 127.6.210.188:46595
I20260924 21:12:40.730201 13825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.723269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:46595 (local address 127.0.0.1:40918)
0924 21:12:40.723543 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:40.723551 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:40.723604 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:40.723759 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:40.723760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:40.724028 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:40.724093 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.724094 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.724984 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.724985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.725855 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.725858 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.726395 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.726396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.726452 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.726983 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:40.726989 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:40.727450 (+ 461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:40.729097 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.729098 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.729099 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.729316 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.729506 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.729507 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.729507 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.729573 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.729861 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:40.729862 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:40.729904 (+ 42us) client_negotiation.cc:769] Sending connection context
0924 21:12:40.729989 (+ 85us) client_negotiation.cc:240] Negotiation successful
0924 21:12:40.730094 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":25,"thread_start_us":80,"threads_started":1}
I20260924 21:12:40.730309 13805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.723268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40918 (local address 127.6.210.188:46595)
0924 21:12:40.723305 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:40.723308 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:40.723588 (+ 280us) server_negotiation.cc:407] Connection header received
0924 21:12:40.723664 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:40.723665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:40.723682 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:40.723708 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:40.724172 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.724896 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.725944 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.726244 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.726321 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.727522 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:40.727523 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:40.727524 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:40.727529 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:40.729014 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.729391 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.729391 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.729392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.729441 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.729660 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.729661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.729661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.729791 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:40.729847 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:40.730015 (+ 168us) server_negotiation.cc:299] Negotiation successful
0924 21:12:40.730168 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":19}
I20260924 21:12:40.750682 13122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.722665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57008 (local address 127.6.210.190:39481)
0924 21:12:40.722711 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:40.722713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:40.723404 (+ 691us) server_negotiation.cc:407] Connection header received
0924 21:12:40.723518 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:40.723519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:40.723533 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:40.723567 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:40.724200 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.725020 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.726342 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.726664 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.726710 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.747725 (+ 21015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:40.747728 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:40.747728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:40.747737 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:40.749334 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.749717 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.749718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.749719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.749776 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.749977 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.749978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.749978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.750154 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:40.750216 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:40.750469 (+ 253us) server_negotiation.cc:299] Negotiation successful
0924 21:12:40.750562 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 21:12:40.750864 13718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.722718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58758 (local address 127.6.210.189:44341)
0924 21:12:40.722756 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:40.722758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:40.723454 (+ 696us) server_negotiation.cc:407] Connection header received
0924 21:12:40.723515 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:40.723516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:40.723529 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:40.723587 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:40.724193 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.725024 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.726342 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.726679 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.726718 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.748025 (+ 21307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:40.748027 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:40.748027 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:40.748036 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:40.749626 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.750064 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.750065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.750065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.750121 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.750332 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.750333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.750333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.750473 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:40.750541 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:40.750702 (+ 161us) server_negotiation.cc:299] Negotiation successful
0924 21:12:40.750778 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 21:12:40.751032 13826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.722811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:44341 (local address 127.0.0.1:58758)
0924 21:12:40.723397 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:40.723406 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:40.723463 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:40.723650 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:40.723650 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:40.723998 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:40.724091 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.724093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.725125 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.725126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.726208 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.726210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.726796 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.726797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.726884 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.746914 (+ 20030us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:40.746919 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:40.747590 (+ 671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:40.749738 (+ 2148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.749739 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.749740 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.749952 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.750194 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.750195 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.750195 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.750258 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.750541 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:40.750542 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:40.750588 (+ 46us) client_negotiation.cc:769] Sending connection context
0924 21:12:40.750648 (+ 60us) client_negotiation.cc:240] Negotiation successful
0924 21:12:40.750914 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"mutex_wait_us":64,"thread_start_us":534,"threads_started":1}
I20260924 21:12:40.751245 13824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.722611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.0.0.1:57008)
0924 21:12:40.723338 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:40.723352 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:40.723428 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:40.723642 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:40.723643 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:40.723962 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:40.724090 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.724093 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.725135 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.725135 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.726208 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.726210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.726796 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.726797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.726884 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.746989 (+ 20105us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:40.746994 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:40.747588 (+ 594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:40.749439 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.749441 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.749442 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.749629 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.749852 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.749852 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.749853 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.749915 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.750244 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:40.750250 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:40.750345 (+ 95us) client_negotiation.cc:769] Sending connection context
0924 21:12:40.750785 (+ 440us) client_negotiation.cc:240] Negotiation successful
0924 21:12:40.751154 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"mutex_wait_us":341,"thread_start_us":81,"threads_started":1}
I20260924 21:12:40.756542 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 13144
Sep 24 21:12:40 dist-test-slave-q276 krb5kdc[13037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284359, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.190@KRBTEST.COM for oryx/127.6.210.188@KRBTEST.COM
I20260924 21:12:40.761193 13639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.753664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:46595 (local address 127.0.0.1:40934)
0924 21:12:40.753722 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:40.753732 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:40.753767 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:40.753889 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:40.753889 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:40.754474 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:40.754522 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.754524 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.755157 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.755158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.755701 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.755703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.756171 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.756172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.756244 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.756639 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:40.756644 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:40.758198 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:40.760096 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.760097 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.760097 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.760317 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.760525 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.760526 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.760526 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.760591 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.760870 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:40.760871 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:40.760918 (+ 47us) client_negotiation.cc:769] Sending connection context
0924 21:12:40.760990 (+ 72us) client_negotiation.cc:240] Negotiation successful
0924 21:12:40.761092 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260924 21:12:40.761538 13805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.753719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40934 (local address 127.6.210.188:46595)
0924 21:12:40.753753 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:40.753755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:40.753763 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:12:40.753827 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:40.753828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:40.753835 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:40.753876 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:40.754595 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.755047 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.755784 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.756091 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.756165 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.758303 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:40.758304 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:40.758305 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:40.758312 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:40.760005 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.760395 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.760396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.760396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.760454 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.760659 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.760660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.760661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.760801 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:40.760869 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:40.761042 (+ 173us) server_negotiation.cc:299] Negotiation successful
0924 21:12:40.761131 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 21:12:40.762084 13082 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.6.210.188:46595: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "faabfe445f404446a8f8eef00b4b6037"
member_type: NON_VOTER
last_known_addr {
host: "127.6.210.188"
port: 46595
}
attrs {
promote: true
}
}
dest_uuid: "34863bba7e8146cab737678d7499c335"
cas_config_opid_index: 5
I20260924 21:12:40.762377 13082 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: NON_VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: true } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: NON_VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: true } }
I20260924 21:12:40.762766 13642 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER faabfe445f404446a8f8eef00b4b6037 (127.6.210.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: NON_VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: true } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: NON_VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: true } } }
I20260924 21:12:40.762979 13776 raft_consensus.cc:3037] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:40.763716 13642 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.6.210.188:46595
I20260924 21:12:40.763787 13642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "34863bba7e8146cab737678d7499c335" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: NON_VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: true } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: NON_VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: true } } }
I20260924 21:12:40.763846 13642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:40.764066 13776 raft_consensus.cc:1261] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [term 1 FOLLOWER]: Refusing update from remote peer 34863bba7e8146cab737678d7499c335: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260924 21:12:40.764228 13734 master.cc:537] Master@127.6.210.188:46595 shutting down...
I20260924 21:12:40.764350 13640 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: NON_VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 21:12:40.765000 13069 consensus_peers.cc:487] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 -> Peer faabfe445f404446a8f8eef00b4b6037 (127.6.210.188:46595): Couldn't send request to peer faabfe445f404446a8f8eef00b4b6037. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260924 21:12:40.766331 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.129:43821
--local_ip_for_outbound_sockets=127.6.210.129
--tserver_master_addrs=127.6.210.190:39481
--webserver_port=41683
--webserver_interface=127.6.210.129
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.129.keytab
--principal=oryx/127.6.210.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:39813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.210.190:39481,127.6.210.189:44341
--tserver_master_addrs=127.6.210.190:39481,127.6.210.189:44341,127.6.210.188:46595 with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:12:40.767936 13734 raft_consensus.cc:2229] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:12:40.768023 13734 raft_consensus.cc:2258] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:40.768064 13734 tablet_replica.cc:330] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037: stopping tablet replica
I20260924 21:12:40.769727 13639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.764033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:44341 (local address 127.0.0.1:58762)
0924 21:12:40.764326 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:40.764595 (+ 269us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:40.764643 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:40.764867 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:40.764868 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:40.764869 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:40.764906 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.764908 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.765803 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.765804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.767755 (+ 1951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.767758 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.769437 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.769438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.769510 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.769534 (+ 24us) client_negotiation.cc:769] Sending connection context
0924 21:12:40.769583 (+ 49us) client_negotiation.cc:240] Negotiation successful
0924 21:12:40.769625 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278}
I20260924 21:12:40.770064 13718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.764093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58762 (local address 127.6.210.189:44341)
0924 21:12:40.764136 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:40.764138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:40.764709 (+ 571us) server_negotiation.cc:407] Connection header received
0924 21:12:40.764737 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:40.764738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:40.764762 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:40.764817 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:40.764972 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.765720 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.767860 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.768460 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.768922 (+ 462us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.769002 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:40.769940 (+ 938us) server_negotiation.cc:299] Negotiation successful
0924 21:12:40.769990 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 21:12:40.770781 13690 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [term 1 LEARNER]: Refusing update from remote peer 34863bba7e8146cab737678d7499c335: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260924 21:12:40.771018 13640 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: NON_VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260924 21:12:40.771612 13690 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER faabfe445f404446a8f8eef00b4b6037 (127.6.210.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: NON_VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: true } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: NON_VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: true } } }
I20260924 21:12:40.773216 13720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34863bba7e8146cab737678d7499c335. Latest consensus state: current_term: 1 leader_uuid: "34863bba7e8146cab737678d7499c335" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: NON_VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: true } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: NON_VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: true } } }
I20260924 21:12:40.773315 13720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [sys.catalog]: This master's current role is: LEARNER
I20260924 21:12:40.773382 13720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "34863bba7e8146cab737678d7499c335" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: NON_VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: true } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: NON_VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: true } } }
I20260924 21:12:40.773444 13720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [sys.catalog]: This master's current role is: LEARNER
I20260924 21:12:40.773718 13640 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335: attempting to promote NON_VOTER 9bbea6a1f4d844e9a0dedfbaa1c60776 to VOTER
I20260924 21:12:40.773936 13640 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: NON_VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: true } }
I20260924 21:12:40.774364 13690 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [term 1 LEARNER]: Refusing update from remote peer 34863bba7e8146cab737678d7499c335: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260924 21:12:40.774416 13639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.774319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:46595 (local address 127.0.0.1:40938)
0924 21:12:40.774353 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:40.774390 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.188:46595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260924 21:12:40.774710 13642 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 21:12:40.774770 13639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.774664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:46595 (local address 127.0.0.1:40944)
0924 21:12:40.774709 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:40.774748 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.188:46595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260924 21:12:40.774845 13069 consensus_peers.cc:487] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 -> Peer faabfe445f404446a8f8eef00b4b6037 (127.6.210.188:46595): Couldn't send request to peer faabfe445f404446a8f8eef00b4b6037. Status: Network error: Client connection negotiation failed: client connection to 127.6.210.188:46595: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 21:12:40.775506 13641 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 9bbea6a1f4d844e9a0dedfbaa1c60776 (127.6.210.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: NON_VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: true } } }
I20260924 21:12:40.775720 13690 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 9bbea6a1f4d844e9a0dedfbaa1c60776 (127.6.210.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: NON_VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: true } } }
I20260924 21:12:40.776386 13641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "34863bba7e8146cab737678d7499c335" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: NON_VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: true } } }
I20260924 21:12:40.776460 13641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:40.776506 13641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "34863bba7e8146cab737678d7499c335" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: NON_VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: true } } }
I20260924 21:12:40.776546 13641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:40.777066 13829 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "34863bba7e8146cab737678d7499c335" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: NON_VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: true } } }
I20260924 21:12:40.777241 13829 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:12:40.781489 13734 master.cc:552] Master@127.6.210.188:46595 shutdown complete.
I20260924 21:12:40.781536 13734 master_runner.cc:305] Clearing existing system tablet
I20260924 21:12:40.781665 13734 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:40.781702 13734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:40.782676 13734 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260924 21:12:40.783234 13831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:40.783383 13734 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:40.783437 13734 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:40.783460 13734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-2/wal
uuid: "faabfe445f404446a8f8eef00b4b6037"
format_stamp: "Formatted at 2026-09-24 21:12:40 on dist-test-slave-q276"
server_key: "9e48d1d9e358962f65102bcca80948bc"
server_key_iv: "4f384b1fed6a538c9f160a8edaced363"
server_key_version: "encryptionkey@0"
I20260924 21:12:40.784329 13734 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:40.785686 13734 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260924 21:12:40.785773 13734 log.cc:1196] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260924 21:12:40.786006 13734 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037: Deleting consensus metadata
I20260924 21:12:40.786157 13734 master_runner.cc:315] Copying system tablet from 127.6.210.190:39481
I20260924 21:12:40.786792 13734 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037: tablet copy: Beginning tablet copy session from remote peer at address 127.6.210.190:39481
I20260924 21:12:40.794157 13122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.787203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57020 (local address 127.6.210.190:39481)
0924 21:12:40.787249 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:40.787252 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:40.787293 (+ 41us) server_negotiation.cc:407] Connection header received
0924 21:12:40.787358 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:40.787359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:40.787368 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:40.787443 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:40.787778 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.788551 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.789599 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.789883 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.789955 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.791285 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:40.791287 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:40.791288 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:40.791295 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:40.792975 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.793363 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.793364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.793364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.793423 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.793637 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.793638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.793638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.793764 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:40.793828 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:40.793991 (+ 163us) server_negotiation.cc:299] Negotiation successful
0924 21:12:40.794066 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 21:12:40.794158 13837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.787118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.0.0.1:57020)
0924 21:12:40.787238 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:40.787249 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:40.787306 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:40.787450 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:40.787451 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:40.787649 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:40.787704 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.787705 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.788650 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.788651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.789503 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.789506 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.789966 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.789967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.790026 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.790787 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:40.790792 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:40.791182 (+ 390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:40.793066 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.793067 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.793067 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.793284 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.793498 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.793499 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.793499 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.793570 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.793833 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:40.793834 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:40.793881 (+ 47us) client_negotiation.cc:769] Sending connection context
0924 21:12:40.793954 (+ 73us) client_negotiation.cc:240] Negotiation successful
0924 21:12:40.794061 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":68,"threads_started":1}
I20260924 21:12:40.794515 13102 tablet_copy_service.cc:140] P 34863bba7e8146cab737678d7499c335: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer faabfe445f404446a8f8eef00b4b6037 ({username='oryx', principal='oryx/127.6.210.188@KRBTEST.COM'} at 127.0.0.1:57020)
I20260924 21:12:40.794561 13102 tablet_copy_service.cc:161] P 34863bba7e8146cab737678d7499c335: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer faabfe445f404446a8f8eef00b4b6037 at {username='oryx', principal='oryx/127.6.210.188@KRBTEST.COM'} at 127.0.0.1:57020: session id = faabfe445f404446a8f8eef00b4b6037-00000000000000000000000000000000
I20260924 21:12:40.795105 13102 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 21:12:40.795596 13734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260924 21:12:40.796882 13734 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037: tablet copy: Starting download of 0 data blocks...
I20260924 21:12:40.797060 13734 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037: tablet copy: Starting download of 1 WAL segments...
I20260924 21:12:40.797827 13734 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 21:12:40.798736 13102 tablet_copy_service.cc:342] P 34863bba7e8146cab737678d7499c335: Request end of tablet copy session faabfe445f404446a8f8eef00b4b6037-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.6.210.188@KRBTEST.COM'} at 127.0.0.1:57020
I20260924 21:12:40.798784 13102 tablet_copy_service.cc:433] P 34863bba7e8146cab737678d7499c335: ending tablet copy session faabfe445f404446a8f8eef00b4b6037-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer faabfe445f404446a8f8eef00b4b6037
I20260924 21:12:40.801286 13734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:40.802580 13838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:40.802745 13734 server_base.cc:1034] running on GCE node
W20260924 21:12:40.806437 13841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:40.807377 13839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:40.807579 13734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:40.807739 13734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:40.808876 13734 hybrid_clock.cc:648] HybridClock initialized: now 1790284360808864 us; error 32 us; skew 500 ppm
Sep 24 21:12:40 dist-test-slave-q276 krb5kdc[13037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284360, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:40.810470 13734 init.cc:377] Logged in from keytab as oryx/127.6.210.188@KRBTEST.COM (short username oryx)
I20260924 21:12:40.810758 13734 webserver.cc:460] Webserver started at http://127.6.210.188:36253/ using document root <none> and password file <none>
I20260924 21:12:40.810884 13734 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:40.810925 13734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:40.811705 13734 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:40.812309 13848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:40.812475 13734 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:12:40.812533 13734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-2/wal
uuid: "faabfe445f404446a8f8eef00b4b6037"
format_stamp: "Formatted at 2026-09-24 21:12:40 on dist-test-slave-q276"
server_key: "9e48d1d9e358962f65102bcca80948bc"
server_key_iv: "4f384b1fed6a538c9f160a8edaced363"
server_key_version: "encryptionkey@0"
I20260924 21:12:40.812597 13734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:40.831749 13734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:40.833665 13734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:40.836809 13734 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.188:46595
I20260924 21:12:40.837163 13734 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/master-2/data/info.pb
I20260924 21:12:40.837411 13900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.188:46595 every 8 connection(s)
I20260924 21:12:40.838178 13901 sys_catalog.cc:263] Verifying existing consensus state
I20260924 21:12:40.838740 13901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037: Bootstrap starting.
I20260924 21:12:40.841729 13901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 21:12:40.841940 13901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037: Bootstrap complete.
I20260924 21:12:40.842121 13901 raft_consensus.cc:348] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: NON_VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: true } }
I20260924 21:12:40.842213 13901 raft_consensus.cc:729] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: faabfe445f404446a8f8eef00b4b6037, State: Initialized, Role: LEARNER
I20260924 21:12:40.842252 13901 consensus_queue.cc:260] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: NON_VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: true } }
I20260924 21:12:40.842480 13901 sys_catalog.cc:564] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:12:40.842461 13904 sys_catalog.cc:455] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: NON_VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: true } } }
I20260924 21:12:40.842664 13904 sys_catalog.cc:458] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [sys.catalog]: This master's current role is: LEARNER
I20260924 21:12:40.844017 13915 catalog_manager.cc:1259] Loaded cluster ID: 91a4d8d24fc44b61b0e4d153ce600838
I20260924 21:12:40.844101 13915 catalog_manager.cc:1552] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037: loading cluster ID for follower catalog manager: success
I20260924 21:12:40.845395 13915 catalog_manager.cc:1574] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037: acquiring CA information for follower catalog manager: success
I20260924 21:12:40.845744 13915 catalog_manager.cc:1602] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260924 21:12:40.865810 13827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:40.866037 13827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:40.866079 13827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:40.867372 13827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:40.867442 13827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:40.867487 13827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.129
I20260924 21:12:40.868961 13827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:39813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.129.keytab
--principal=oryx/127.6.210.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.129:43821
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.210.129
--webserver_port=41683
--tserver_master_addrs=127.6.210.190:39481,127.6.210.189:44341,127.6.210.188:46595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.129
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:40.869230 13827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:40.869457 13827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:40.874497 13921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:40.874505 13920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:40.874584 13923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:40.875129 13827 server_base.cc:1034] running on GCE node
I20260924 21:12:40.875365 13827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:40.875653 13827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:40.876818 13827 hybrid_clock.cc:648] HybridClock initialized: now 1790284360876807 us; error 26 us; skew 500 ppm
Sep 24 21:12:40 dist-test-slave-q276 krb5kdc[13037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284360, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:40.879500 13827 init.cc:377] Logged in from keytab as oryx/127.6.210.129@KRBTEST.COM (short username oryx)
I20260924 21:12:40.880030 13827 webserver.cc:460] Webserver started at http://127.6.210.129:41683/ using document root <none> and password file <none>
I20260924 21:12:40.880265 13827 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:40.880345 13827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:40.881718 13827 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:40.882310 13930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:40.882488 13827 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:12:40.882584 13827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-0/wal
uuid: "cb42303b63fc4ae2a46db49707af5edb"
format_stamp: "Formatted at 2026-09-24 21:12:39 on dist-test-slave-q276"
server_key: "efd49ef44cfad244533e46ab99e448ac"
server_key_iv: "f43947661c5201095c3bf7e087243625"
server_key_version: "encryptionkey@0"
I20260924 21:12:40.882942 13827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:40.906908 13827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:40.909412 13827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:40.909585 13827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:40.909811 13827 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:40.910112 13827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:40.910145 13827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:40.910167 13827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:40.910179 13827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:40.916998 13827 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.129:43821
I20260924 21:12:40.917043 14043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.129:43821 every 8 connection(s)
I20260924 21:12:40.917388 13827 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
I20260924 21:12:40.922043 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 13827
I20260924 21:12:40.922188 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 13278
Sep 24 21:12:40 dist-test-slave-q276 krb5kdc[13037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284360, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.188@KRBTEST.COM
Sep 24 21:12:40 dist-test-slave-q276 krb5kdc[13037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284360, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.189@KRBTEST.COM
Sep 24 21:12:40 dist-test-slave-q276 krb5kdc[13037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284360, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM
I20260924 21:12:40.929569 13718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.918148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:42325 (local address 127.6.210.189:44341)
0924 21:12:40.918253 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:40.918255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:40.918486 (+ 231us) server_negotiation.cc:407] Connection header received
0924 21:12:40.919544 (+ 1058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:40.919546 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:40.919558 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:40.919587 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:40.920468 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.921214 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.922249 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.922573 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.922612 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.925755 (+ 3143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:40.925757 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:40.925757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:40.925765 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:40.927813 (+ 2048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.928439 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.928441 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.928441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.928499 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.928781 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.928782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.928783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.928943 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:40.928985 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:40.929371 (+ 386us) server_negotiation.cc:299] Negotiation successful
0924 21:12:40.929464 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260924 21:12:40.929665 13122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.918204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:41121 (local address 127.6.210.190:39481)
0924 21:12:40.918309 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:40.918311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:40.918640 (+ 329us) server_negotiation.cc:407] Connection header received
0924 21:12:40.919531 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:40.919532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:40.919545 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:40.919605 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:40.920453 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.920944 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.922064 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.922414 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.922458 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.926826 (+ 4368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:40.926828 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:40.926828 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:40.926836 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:40.928516 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.928906 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.928907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.928907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.928959 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.929150 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.929151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.929151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.929283 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:40.929325 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:40.929508 (+ 183us) server_negotiation.cc:299] Negotiation successful
0924 21:12:40.929578 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260924 21:12:40.929728 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.130:34739
--local_ip_for_outbound_sockets=127.6.210.130
--tserver_master_addrs=127.6.210.190:39481
--webserver_port=45343
--webserver_interface=127.6.210.130
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.130.keytab
--principal=oryx/127.6.210.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:39813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.210.190:39481,127.6.210.189:44341
--tserver_master_addrs=127.6.210.190:39481,127.6.210.189:44341,127.6.210.188:46595 with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:12:40.929781 14048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.918068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:46595 (local address 127.6.210.129:40225)
0924 21:12:40.918344 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:40.918389 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:40.919417 (+ 1028us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:40.919773 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:40.919774 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:40.920173 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:40.920359 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.920365 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.921546 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.921546 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.922503 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.922505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.922997 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.922998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.923082 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.923544 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:40.923549 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:40.926096 (+ 2547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:40.928435 (+ 2339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.928436 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.928436 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.928643 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.928861 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.928861 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.928861 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.928915 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.929201 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:40.929202 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:40.929241 (+ 39us) client_negotiation.cc:769] Sending connection context
0924 21:12:40.929331 (+ 90us) client_negotiation.cc:240] Negotiation successful
0924 21:12:40.929523 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":8,"thread_start_us":87,"threads_started":1}
I20260924 21:12:40.929949 14049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.918068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:44341 (local address 127.6.210.129:42325)
0924 21:12:40.918390 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:40.918399 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:40.919417 (+ 1018us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:40.919685 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:40.919689 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:40.920179 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:40.920359 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.920365 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.921304 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.921304 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.922145 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.922148 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.922690 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.922691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.922821 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.923391 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:40.923399 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:40.925652 (+ 2253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:40.927995 (+ 2343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.928003 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.928005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.928338 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.928569 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.928569 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.928570 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.928709 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.929031 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:40.929037 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:40.929221 (+ 184us) client_negotiation.cc:769] Sending connection context
0924 21:12:40.929691 (+ 470us) client_negotiation.cc:240] Negotiation successful
0924 21:12:40.929801 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":33,"thread_start_us":94,"threads_started":1}
I20260924 21:12:40.930419 14046 heartbeater.cc:347] Connected to a master server at 127.6.210.189:44341
I20260924 21:12:40.930511 14046 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:40.930702 14046 heartbeater.cc:511] Master 127.6.210.189:44341 requested a full tablet report, sending...
I20260924 21:12:40.930904 14051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.918068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.6.210.129:41121)
0924 21:12:40.918571 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:40.918580 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:40.919426 (+ 846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:40.919695 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:40.919696 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:40.920166 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:40.920358 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.920363 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.921048 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.921049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.921957 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:40.921959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.922881 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.922882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:40.922955 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.923858 (+ 903us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:40.923864 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:40.926709 (+ 2845us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:40.928610 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.928611 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.928612 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.928824 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.929031 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:40.929032 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:40.929032 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:40.929079 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:40.929370 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:40.929371 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:40.929403 (+ 32us) client_negotiation.cc:769] Sending connection context
0924 21:12:40.930660 (+ 1257us) client_negotiation.cc:240] Negotiation successful
0924 21:12:40.930810 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"mutex_wait_us":209,"thread_start_us":59,"threads_started":1}
I20260924 21:12:40.931221 13680 ts_manager.cc:194] Registered new tserver with Master: cb42303b63fc4ae2a46db49707af5edb (127.6.210.129:43821)
I20260924 21:12:40.931399 14050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:40.918118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:40225 (local address 127.6.210.188:46595)
0924 21:12:40.918253 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:40.918255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:40.918486 (+ 231us) server_negotiation.cc:407] Connection header received
0924 21:12:40.919668 (+ 1182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:40.919669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:40.919682 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:40.919750 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:40.920468 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.921466 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.922584 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:40.922881 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:40.922945 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:40.926194 (+ 3249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:40.926195 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:40.926196 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:40.926204 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:40.928340 (+ 2136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.928735 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.928736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.928737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.928793 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:40.928982 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:40.928983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:40.928983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:40.929117 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:40.929175 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:40.930079 (+ 904us) server_negotiation.cc:299] Negotiation successful
0924 21:12:40.930309 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":66,"threads_started":1}
I20260924 21:12:40.931608 14045 heartbeater.cc:347] Connected to a master server at 127.6.210.190:39481
I20260924 21:12:40.931667 14045 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:40.931702 14044 heartbeater.cc:347] Connected to a master server at 127.6.210.188:46595
I20260924 21:12:40.931747 14044 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:40.931824 14044 heartbeater.cc:511] Master 127.6.210.188:46595 requested a full tablet report, sending...
I20260924 21:12:40.931737 14045 heartbeater.cc:511] Master 127.6.210.190:39481 requested a full tablet report, sending...
I20260924 21:12:40.932142 13082 ts_manager.cc:194] Re-registered known tserver with Master: cb42303b63fc4ae2a46db49707af5edb (127.6.210.129:43821)
I20260924 21:12:40.932271 13857 ts_manager.cc:194] Registered new tserver with Master: cb42303b63fc4ae2a46db49707af5edb (127.6.210.129:43821)
I20260924 21:12:40.932682 13082 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.210.129@KRBTEST.COM'} at 127.6.210.129:41121
W20260924 21:12:41.012197 14052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:41.012514 14052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:41.012544 14052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:41.013845 14052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:41.013886 14052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:41.013937 14052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.130
I20260924 21:12:41.015493 14052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:39813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.130.keytab
--principal=oryx/127.6.210.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.130:34739
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.210.130
--webserver_port=45343
--tserver_master_addrs=127.6.210.190:39481,127.6.210.189:44341,127.6.210.188:46595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.130
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:41.015678 14052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:41.015862 14052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:41.020925 14057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:41.020988 14060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:41.021085 14052 server_base.cc:1034] running on GCE node
W20260924 21:12:41.020951 14058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:41.021355 14052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:41.021642 14052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:41.022809 14052 hybrid_clock.cc:648] HybridClock initialized: now 1790284361022803 us; error 36 us; skew 500 ppm
Sep 24 21:12:41 dist-test-slave-q276 krb5kdc[13037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284361, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:41.025259 14052 init.cc:377] Logged in from keytab as oryx/127.6.210.130@KRBTEST.COM (short username oryx)
I20260924 21:12:41.025738 14052 webserver.cc:460] Webserver started at http://127.6.210.130:45343/ using document root <none> and password file <none>
I20260924 21:12:41.025971 14052 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:41.026053 14052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:41.027546 14052 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:41.028184 14067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:41.028334 14052 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:12:41.028430 14052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-1/wal
uuid: "d0982fc9b1f040af82515f52e5400c5f"
format_stamp: "Formatted at 2026-09-24 21:12:40 on dist-test-slave-q276"
server_key: "32a2c1e23f5b5aa4334cd9f50101804b"
server_key_iv: "2df42268f4c6077bf3160dbb04f9f22c"
server_key_version: "encryptionkey@0"
I20260924 21:12:41.028789 14052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:41.037177 14052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:41.039058 14052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:41.039224 14052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:41.039454 14052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:41.039757 14052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:41.039809 14052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:41.039850 14052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:41.039880 14052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:41.046268 14052 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.130:34739
I20260924 21:12:41.046334 14180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.130:34739 every 8 connection(s)
I20260924 21:12:41.046681 14052 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
Sep 24 21:12:41 dist-test-slave-q276 krb5kdc[13037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284361, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM
Sep 24 21:12:41 dist-test-slave-q276 krb5kdc[13037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284361, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.189@KRBTEST.COM
Sep 24 21:12:41 dist-test-slave-q276 krb5kdc[13037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284361, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.188@KRBTEST.COM
I20260924 21:12:41.055444 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 14052
I20260924 21:12:41.055613 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 13412
I20260924 21:12:41.058518 13122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:41.047382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:55325 (local address 127.6.210.190:39481)
0924 21:12:41.047896 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:41.047898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:41.047910 (+ 12us) server_negotiation.cc:407] Connection header received
0924 21:12:41.048745 (+ 835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:41.048747 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:41.048759 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:41.048816 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:41.049639 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.050424 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.051603 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.051877 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.051940 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:41.054655 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:41.054656 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:41.054657 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:41.054664 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:41.056893 (+ 2229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:41.057308 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:41.057309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:41.057309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:41.057375 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:41.057711 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:41.057712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:41.057713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:41.057837 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:41.057919 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:41.058243 (+ 324us) server_negotiation.cc:299] Negotiation successful
0924 21:12:41.058369 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480}
I20260924 21:12:41.058990 14185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:41.047404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.6.210.130:55325)
0924 21:12:41.047575 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:41.047610 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:41.048634 (+ 1024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:41.048879 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:41.048883 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:41.049337 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:41.049533 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:41.049539 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.050500 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.050501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:41.051490 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:41.051493 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.051990 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.051991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:41.052052 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:41.052523 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:41.052528 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:41.054539 (+ 2011us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:41.056991 (+ 2452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:41.056997 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:41.057000 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:41.057229 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:41.057471 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:41.057472 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:41.057473 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:41.057621 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:41.057910 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:41.057917 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:41.058085 (+ 168us) client_negotiation.cc:769] Sending connection context
0924 21:12:41.058220 (+ 135us) client_negotiation.cc:240] Negotiation successful
0924 21:12:41.058423 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":51,"thread_start_us":95,"threads_started":1}
I20260924 21:12:41.060094 14187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:41.047404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:44341 (local address 127.6.210.130:32889)
0924 21:12:41.047931 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:41.047942 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:41.048653 (+ 711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:41.048879 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:41.048883 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:41.049334 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:41.049533 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:41.049539 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.050518 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.050519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:41.051314 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:41.051316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.051761 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.051761 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:41.051894 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:41.052415 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:41.052423 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:41.057001 (+ 4578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:41.058986 (+ 1985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:41.058987 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:41.058988 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:41.059255 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:41.059445 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:41.059446 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:41.059446 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:41.059504 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:41.059761 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:41.059762 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:41.059806 (+ 44us) client_negotiation.cc:769] Sending connection context
0924 21:12:41.059889 (+ 83us) client_negotiation.cc:240] Negotiation successful
0924 21:12:41.059993 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"mutex_wait_us":62,"thread_start_us":96,"threads_started":1}
I20260924 21:12:41.060083 13718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:41.047438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:32889 (local address 127.6.210.189:44341)
0924 21:12:41.047482 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:41.047484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:41.048011 (+ 527us) server_negotiation.cc:407] Connection header received
0924 21:12:41.048740 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:41.048741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:41.048754 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:41.048803 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:41.049640 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.050420 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.051407 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.051682 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.051748 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:41.057110 (+ 5362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:41.057111 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:41.057112 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:41.057119 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:41.058902 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:41.059323 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:41.059324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:41.059324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:41.059389 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:41.059579 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:41.059580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:41.059581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:41.059693 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:41.059769 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:41.059925 (+ 156us) server_negotiation.cc:299] Negotiation successful
0924 21:12:41.059996 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 21:12:41.060494 14183 heartbeater.cc:347] Connected to a master server at 127.6.210.189:44341
I20260924 21:12:41.060597 14183 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:41.060794 14183 heartbeater.cc:511] Master 127.6.210.189:44341 requested a full tablet report, sending...
I20260924 21:12:41.061241 13680 ts_manager.cc:194] Registered new tserver with Master: d0982fc9b1f040af82515f52e5400c5f (127.6.210.130:34739)
I20260924 21:12:41.062213 14182 heartbeater.cc:347] Connected to a master server at 127.6.210.190:39481
I20260924 21:12:41.062340 14182 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:41.062429 14182 heartbeater.cc:511] Master 127.6.210.190:39481 requested a full tablet report, sending...
I20260924 21:12:41.062463 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.131:37515
--local_ip_for_outbound_sockets=127.6.210.131
--tserver_master_addrs=127.6.210.190:39481
--webserver_port=45533
--webserver_interface=127.6.210.131
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.131.keytab
--principal=oryx/127.6.210.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:39813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.210.190:39481,127.6.210.189:44341
--tserver_master_addrs=127.6.210.190:39481,127.6.210.189:44341,127.6.210.188:46595 with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:12:41.063927 13082 ts_manager.cc:194] Re-registered known tserver with Master: d0982fc9b1f040af82515f52e5400c5f (127.6.210.130:34739)
I20260924 21:12:41.064256 14186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:41.047766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:46595 (local address 127.6.210.130:41187)
0924 21:12:41.048843 (+ 1077us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:41.048852 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:41.048910 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:41.049029 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:41.049029 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:41.049333 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:41.049531 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:41.049536 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.050346 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.050346 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:41.051491 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:41.051493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.051957 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.051958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:41.052025 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:41.052715 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:41.052720 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:41.055531 (+ 2811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:41.063113 (+ 7582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:41.063114 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:41.063115 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:41.063349 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:41.063543 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:41.063544 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:41.063544 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:41.063603 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:41.063891 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:41.063892 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:41.063934 (+ 42us) client_negotiation.cc:769] Sending connection context
0924 21:12:41.064022 (+ 88us) client_negotiation.cc:240] Negotiation successful
0924 21:12:41.064153 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1061,"mutex_wait_us":62,"thread_start_us":65,"threads_started":1}
I20260924 21:12:41.064267 14050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:41.047853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:41187 (local address 127.6.210.188:46595)
0924 21:12:41.047901 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:41.047903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:41.048892 (+ 989us) server_negotiation.cc:407] Connection header received
0924 21:12:41.048967 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:41.048968 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:41.048981 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:41.049027 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:41.049625 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.050172 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.051574 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.051866 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.051954 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:41.055644 (+ 3690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:41.055646 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:41.055646 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:41.055655 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:41.063027 (+ 7372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:41.063426 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:41.063426 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:41.063427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:41.063474 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:41.063680 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:41.063681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:41.063681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:41.063827 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:41.063886 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:41.064071 (+ 185us) server_negotiation.cc:299] Negotiation successful
0924 21:12:41.064177 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 21:12:41.064450 13082 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.210.130@KRBTEST.COM'} at 127.6.210.130:55325
I20260924 21:12:41.064949 14181 heartbeater.cc:347] Connected to a master server at 127.6.210.188:46595
I20260924 21:12:41.065016 14181 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:41.065141 14181 heartbeater.cc:511] Master 127.6.210.188:46595 requested a full tablet report, sending...
I20260924 21:12:41.065459 13857 ts_manager.cc:194] Registered new tserver with Master: d0982fc9b1f040af82515f52e5400c5f (127.6.210.130:34739)
W20260924 21:12:41.144994 14188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:41.145200 14188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:41.145222 14188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:41.146400 14188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:41.146442 14188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:41.146497 14188 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.131
I20260924 21:12:41.147965 14188 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:39813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.131.keytab
--principal=oryx/127.6.210.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.131:37515
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.210.131
--webserver_port=45533
--tserver_master_addrs=127.6.210.190:39481,127.6.210.189:44341,127.6.210.188:46595
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.131
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:41.148211 14188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:41.148447 14188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:41.153379 14196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:41.153614 14193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:41.153620 14194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:41.153802 14188 server_base.cc:1034] running on GCE node
I20260924 21:12:41.154062 14188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:41.154379 14188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:41.155516 14188 hybrid_clock.cc:648] HybridClock initialized: now 1790284361155508 us; error 27 us; skew 500 ppm
Sep 24 21:12:41 dist-test-slave-q276 krb5kdc[13037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284361, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:41.157943 14188 init.cc:377] Logged in from keytab as oryx/127.6.210.131@KRBTEST.COM (short username oryx)
I20260924 21:12:41.158449 14188 webserver.cc:460] Webserver started at http://127.6.210.131:45533/ using document root <none> and password file <none>
I20260924 21:12:41.158627 14188 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:41.158684 14188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:41.159914 14188 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:41.160522 14203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:41.160730 14188 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:41.160831 14188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-2/wal
uuid: "5ecda7f62fe647d78e4a990968bf5327"
format_stamp: "Formatted at 2026-09-24 21:12:40 on dist-test-slave-q276"
server_key: "351779a6770d3cfa2cb6ae624833bd81"
server_key_iv: "e2d35f62712e7e24298495636e1bd353"
server_key_version: "encryptionkey@0"
I20260924 21:12:41.161178 14188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:41.171049 14188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:41.173061 14188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:41.173287 14188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:41.173532 14188 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:41.173832 14188 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:41.173887 14188 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:41.173928 14188 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:41.173954 14188 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:41.179965 14188 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.131:37515
I20260924 21:12:41.179994 14316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.131:37515 every 8 connection(s)
I20260924 21:12:41.180346 14188 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
Sep 24 21:12:41 dist-test-slave-q276 krb5kdc[13037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284361, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.189@KRBTEST.COM
Sep 24 21:12:41 dist-test-slave-q276 krb5kdc[13037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284361, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM
Sep 24 21:12:41 dist-test-slave-q276 krb5kdc[13037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284361, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.188@KRBTEST.COM
I20260924 21:12:41.188036 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 14188
I20260924 21:12:41.190867 13718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:41.181101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:33417 (local address 127.6.210.189:44341)
0924 21:12:41.181147 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:41.181150 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:41.181533 (+ 383us) server_negotiation.cc:407] Connection header received
0924 21:12:41.182076 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:41.182078 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:41.182089 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:41.182175 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:41.182946 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.183426 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.184440 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.184631 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.184717 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:41.187215 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:41.187216 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:41.187216 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:41.187224 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:41.189284 (+ 2060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:41.189777 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:41.189778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:41.189778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:41.189833 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:41.190105 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:41.190106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:41.190106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:41.190238 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:41.190315 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:41.190695 (+ 380us) server_negotiation.cc:299] Negotiation successful
0924 21:12:41.190771 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 21:12:41.191143 14050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:41.181163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:45269 (local address 127.6.210.188:46595)
0924 21:12:41.181203 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:41.181206 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:41.181501 (+ 295us) server_negotiation.cc:407] Connection header received
0924 21:12:41.182099 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:41.182101 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:41.182113 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:41.182146 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:41.182945 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.183698 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.184838 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.185086 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.185156 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:41.188127 (+ 2971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:41.188128 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:41.188129 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:41.188137 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:41.189951 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:41.190355 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:41.190357 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:41.190357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:41.190408 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:41.190600 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:41.190601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:41.190601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:41.190724 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:41.190774 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:41.190969 (+ 195us) server_negotiation.cc:299] Negotiation successful
0924 21:12:41.191050 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 21:12:41.191414 13122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:41.181307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:51569 (local address 127.6.210.190:39481)
0924 21:12:41.181360 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:41.181362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:41.181669 (+ 307us) server_negotiation.cc:407] Connection header received
0924 21:12:41.182100 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:41.182101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:41.182113 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:41.182145 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:41.182945 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.183700 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.184803 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.185105 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.185173 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:41.188119 (+ 2946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:41.188120 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:41.188121 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:41.188130 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:41.190143 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:41.190564 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:41.190565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:41.190565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:41.190628 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:41.190842 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:41.190843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:41.190843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:41.190993 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:41.191052 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:41.191234 (+ 182us) server_negotiation.cc:299] Negotiation successful
0924 21:12:41.191329 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 21:12:41.191409 14321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:41.181045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:44341 (local address 127.6.210.131:33417)
0924 21:12:41.181388 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:41.181408 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:41.181983 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:41.182244 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:41.182248 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:41.182650 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:41.182836 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:41.182840 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.183527 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.183528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:41.184346 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:41.184348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.184724 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.184725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:41.184860 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:41.185469 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:41.185479 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:41.187108 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:41.189374 (+ 2266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:41.189381 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:41.189384 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:41.189682 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:41.189906 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:41.189906 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:41.189906 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:41.190029 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:41.190361 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:41.190367 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:41.190528 (+ 161us) client_negotiation.cc:769] Sending connection context
0924 21:12:41.191015 (+ 487us) client_negotiation.cc:240] Negotiation successful
0924 21:12:41.191179 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":3,"thread_start_us":79,"threads_started":1}
I20260924 21:12:41.191409 14323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:41.181045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:46595 (local address 127.6.210.131:45269)
0924 21:12:41.181363 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:41.181407 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:41.181983 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:41.182246 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:41.182249 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:41.182659 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:41.182837 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:41.182844 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.183794 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.183794 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:41.184731 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:41.184733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.185162 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.185164 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:41.185227 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:41.185627 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:41.185631 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:41.188018 (+ 2387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:41.190025 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:41.190025 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:41.190026 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:41.190230 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:41.190477 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:41.190478 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:41.190478 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:41.190529 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:41.190805 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:41.190806 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:41.190842 (+ 36us) client_negotiation.cc:769] Sending connection context
0924 21:12:41.191214 (+ 372us) client_negotiation.cc:240] Negotiation successful
0924 21:12:41.191292 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":25,"thread_start_us":88,"threads_started":1}
I20260924 21:12:41.191780 14322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:41.181046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.6.210.131:51569)
0924 21:12:41.181606 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:41.181613 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:41.181988 (+ 375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:41.182649 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:41.182650 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:41.182783 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:41.182849 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:41.182851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.183794 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.183794 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:41.184734 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:41.184735 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.185223 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.185224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:41.185280 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:41.185783 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:41.185787 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:41.188008 (+ 2221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:41.190234 (+ 2226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:41.190235 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:41.190235 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:41.190480 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:41.190703 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:41.190704 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:41.190704 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:41.190764 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:41.191088 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:41.191089 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:41.191126 (+ 37us) client_negotiation.cc:769] Sending connection context
0924 21:12:41.191526 (+ 400us) client_negotiation.cc:240] Negotiation successful
0924 21:12:41.191647 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"mutex_wait_us":261,"thread_start_us":57,"threads_started":1}
I20260924 21:12:41.192025 14319 heartbeater.cc:347] Connected to a master server at 127.6.210.189:44341
I20260924 21:12:41.192078 14318 heartbeater.cc:347] Connected to a master server at 127.6.210.190:39481
I20260924 21:12:41.192130 14319 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:41.192154 14317 heartbeater.cc:347] Connected to a master server at 127.6.210.188:46595
I20260924 21:12:41.192183 14317 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:41.192330 14317 heartbeater.cc:511] Master 127.6.210.188:46595 requested a full tablet report, sending...
I20260924 21:12:41.192130 14318 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:41.192586 14318 heartbeater.cc:511] Master 127.6.210.190:39481 requested a full tablet report, sending...
I20260924 21:12:41.192330 14319 heartbeater.cc:511] Master 127.6.210.189:44341 requested a full tablet report, sending...
I20260924 21:12:41.192775 13857 ts_manager.cc:194] Registered new tserver with Master: 5ecda7f62fe647d78e4a990968bf5327 (127.6.210.131:37515)
I20260924 21:12:41.192792 13082 ts_manager.cc:194] Re-registered known tserver with Master: 5ecda7f62fe647d78e4a990968bf5327 (127.6.210.131:37515)
I20260924 21:12:41.193280 13082 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.210.131@KRBTEST.COM'} at 127.6.210.131:51569
I20260924 21:12:41.193732 13680 ts_manager.cc:194] Registered new tserver with Master: 5ecda7f62fe647d78e4a990968bf5327 (127.6.210.131:37515)
I20260924 21:12:41.199723 13718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:41.190096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58768 (local address 127.6.210.189:44341)
0924 21:12:41.190955 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:41.190957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:41.191940 (+ 983us) server_negotiation.cc:407] Connection header received
0924 21:12:41.192036 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:41.192037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:41.192046 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:41.192072 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:41.192988 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.193649 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.194817 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.195093 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.195155 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:41.196670 (+ 1515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:41.196671 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:41.196672 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:41.196680 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:41.198422 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:41.198852 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:41.198854 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:41.198854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:41.198908 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:41.199141 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:41.199142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:41.199142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:41.199279 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:41.199377 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:41.199511 (+ 134us) server_negotiation.cc:299] Negotiation successful
0924 21:12:41.199586 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":846,"thread_start_us":51,"threads_started":1}
I20260924 21:12:41.200522 14330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:41.189601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57032 (local address 127.6.210.190:39481)
0924 21:12:41.190729 (+ 1128us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:41.190731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:41.190887 (+ 156us) server_negotiation.cc:407] Connection header received
0924 21:12:41.190905 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:41.190905 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:41.191438 (+ 533us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:41.193410 (+ 1972us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:41.194054 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.194698 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.195685 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.195985 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.196054 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:41.197427 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:41.197429 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:41.197429 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:41.197435 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:41.199216 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:41.199587 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:41.199588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:41.199588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:41.199642 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:41.200084 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:41.200085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:41.200086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:41.200211 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:41.200281 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:41.200385 (+ 104us) server_negotiation.cc:299] Negotiation successful
0924 21:12:41.200442 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1102,"thread_start_us":90,"threads_started":1}
I20260924 21:12:41.200793 14328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:41.189217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40948 (local address 127.6.210.188:46595)
0924 21:12:41.190385 (+ 1168us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:41.190387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:41.190565 (+ 178us) server_negotiation.cc:407] Connection header received
0924 21:12:41.190585 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:41.190586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:41.190599 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:41.190658 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:41.194154 (+ 3496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.194883 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.195707 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.195909 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.196007 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:41.197824 (+ 1817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:41.197826 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:41.197826 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:41.197833 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:41.199579 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:41.199951 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:41.199953 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:41.199953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:41.200019 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:41.200202 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:41.200203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:41.200204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:41.200324 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:41.200411 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:41.200596 (+ 185us) server_negotiation.cc:299] Negotiation successful
0924 21:12:41.200689 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1148,"thread_start_us":51,"threads_started":1}
I20260924 21:12:41.201924 13082 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57032:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:12:41.203790 13082 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:12:41.211027 14338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:41.207370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:43821 (local address 127.0.0.1:35272)
0924 21:12:41.207502 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:41.207511 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:41.207546 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:41.207924 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:41.207925 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:41.207928 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:41.208011 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:41.208013 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.208961 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.208962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:41.209850 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:41.209853 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.210802 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.210803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:41.210864 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:41.210868 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:12:41.210915 (+ 47us) client_negotiation.cc:240] Negotiation successful
0924 21:12:41.210952 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":82,"threads_started":1}
I20260924 21:12:41.211328 14341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:41.207585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35272 (local address 127.6.210.129:43821)
0924 21:12:41.207762 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:41.207768 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:41.207779 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:12:41.207826 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:41.207828 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:41.207846 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:41.207927 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:41.208099 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.208861 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.209962 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.210726 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.210794 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:41.211152 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:41.211187 (+ 35us) server_negotiation.cc:299] Negotiation successful
0924 21:12:41.211252 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":113,"threads_started":1}
I20260924 21:12:41.211440 13639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:41.207332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:34739 (local address 127.0.0.1:58828)
0924 21:12:41.207368 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:41.207380 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:41.207422 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:41.208214 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:41.208215 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:41.208216 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:41.208257 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:41.208259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.209509 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.209510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:41.210419 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:41.210421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.211238 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.211239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:41.211298 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:41.211301 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:12:41.211335 (+ 34us) client_negotiation.cc:240] Negotiation successful
0924 21:12:41.211363 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260924 21:12:41.211937 14339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:41.207417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58828 (local address 127.6.210.130:34739)
0924 21:12:41.208033 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:41.208038 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:41.208048 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:12:41.208088 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:41.208090 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:41.208106 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:41.208329 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:41.208386 (+ 57us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.209433 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.210499 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.211149 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.211545 (+ 396us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:41.211625 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:41.211657 (+ 32us) server_negotiation.cc:299] Negotiation successful
0924 21:12:41.211717 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":70,"threads_started":1}
I20260924 21:12:41.212186 13978 tablet_service.cc:1467] Processing CreateTablet for tablet 4b2681254f3242eeb2e37d2423452435 (DEFAULT_TABLE table=test-table [id=f4ce7e94c0474e94a3bd0f83b57dc9d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:41.212419 14340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:41.207418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:37515 (local address 127.0.0.1:46712)
0924 21:12:41.207793 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:41.207803 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:41.207914 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:41.208208 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:41.208209 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:41.208211 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:41.208244 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:41.208245 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.209493 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.209494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:41.210681 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:41.210684 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.212144 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.212144 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:41.212208 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:41.212212 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:12:41.212272 (+ 60us) client_negotiation.cc:240] Negotiation successful
0924 21:12:41.212343 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":52,"thread_start_us":73,"threads_started":1}
I20260924 21:12:41.212435 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:41.207595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46712 (local address 127.6.210.131:37515)
0924 21:12:41.208051 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:41.208056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:41.208065 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:12:41.208098 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:41.208100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:41.208113 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:41.208211 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:41.208306 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.209391 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.210784 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.211846 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.211930 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:41.212086 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:41.212290 (+ 204us) server_negotiation.cc:299] Negotiation successful
0924 21:12:41.212332 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":117,"threads_started":1}
I20260924 21:12:41.212565 13978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b2681254f3242eeb2e37d2423452435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:41.212560 14115 tablet_service.cc:1467] Processing CreateTablet for tablet 4b2681254f3242eeb2e37d2423452435 (DEFAULT_TABLE table=test-table [id=f4ce7e94c0474e94a3bd0f83b57dc9d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:41.212875 14115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b2681254f3242eeb2e37d2423452435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:41.213160 14251 tablet_service.cc:1467] Processing CreateTablet for tablet 4b2681254f3242eeb2e37d2423452435 (DEFAULT_TABLE table=test-table [id=f4ce7e94c0474e94a3bd0f83b57dc9d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:41.213498 14251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b2681254f3242eeb2e37d2423452435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:41.216475 14343 tablet_bootstrap.cc:492] T 4b2681254f3242eeb2e37d2423452435 P 5ecda7f62fe647d78e4a990968bf5327: Bootstrap starting.
I20260924 21:12:41.216763 14344 tablet_bootstrap.cc:492] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f: Bootstrap starting.
I20260924 21:12:41.217316 14343 tablet_bootstrap.cc:654] T 4b2681254f3242eeb2e37d2423452435 P 5ecda7f62fe647d78e4a990968bf5327: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:41.217505 14344 tablet_bootstrap.cc:654] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:41.217633 14343 log.cc:824] T 4b2681254f3242eeb2e37d2423452435 P 5ecda7f62fe647d78e4a990968bf5327: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:41.217794 14344 log.cc:824] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:41.217980 14345 tablet_bootstrap.cc:492] T 4b2681254f3242eeb2e37d2423452435 P cb42303b63fc4ae2a46db49707af5edb: Bootstrap starting.
I20260924 21:12:41.218523 14343 tablet_bootstrap.cc:492] T 4b2681254f3242eeb2e37d2423452435 P 5ecda7f62fe647d78e4a990968bf5327: No bootstrap required, opened a new log
I20260924 21:12:41.218542 14344 tablet_bootstrap.cc:492] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f: No bootstrap required, opened a new log
I20260924 21:12:41.218596 14344 ts_tablet_manager.cc:1397] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:12:41.218596 14343 ts_tablet_manager.cc:1397] T 4b2681254f3242eeb2e37d2423452435 P 5ecda7f62fe647d78e4a990968bf5327: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:12:41.220561 14344 raft_consensus.cc:348] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0982fc9b1f040af82515f52e5400c5f" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34739 } } peers { permanent_uuid: "cb42303b63fc4ae2a46db49707af5edb" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43821 } } peers { permanent_uuid: "5ecda7f62fe647d78e4a990968bf5327" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37515 } }
I20260924 21:12:41.220745 14344 raft_consensus.cc:374] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:41.220783 14344 raft_consensus.cc:729] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0982fc9b1f040af82515f52e5400c5f, State: Initialized, Role: FOLLOWER
I20260924 21:12:41.220901 14344 consensus_queue.cc:260] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0982fc9b1f040af82515f52e5400c5f" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34739 } } peers { permanent_uuid: "cb42303b63fc4ae2a46db49707af5edb" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43821 } } peers { permanent_uuid: "5ecda7f62fe647d78e4a990968bf5327" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37515 } }
I20260924 21:12:41.221001 14343 raft_consensus.cc:348] T 4b2681254f3242eeb2e37d2423452435 P 5ecda7f62fe647d78e4a990968bf5327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0982fc9b1f040af82515f52e5400c5f" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34739 } } peers { permanent_uuid: "cb42303b63fc4ae2a46db49707af5edb" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43821 } } peers { permanent_uuid: "5ecda7f62fe647d78e4a990968bf5327" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37515 } }
I20260924 21:12:41.221151 14344 ts_tablet_manager.cc:1428] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260924 21:12:41.221150 14343 raft_consensus.cc:374] T 4b2681254f3242eeb2e37d2423452435 P 5ecda7f62fe647d78e4a990968bf5327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:41.221192 14182 heartbeater.cc:503] Master 127.6.210.190:39481 was elected leader, sending a full tablet report...
I20260924 21:12:41.221197 14343 raft_consensus.cc:729] T 4b2681254f3242eeb2e37d2423452435 P 5ecda7f62fe647d78e4a990968bf5327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ecda7f62fe647d78e4a990968bf5327, State: Initialized, Role: FOLLOWER
I20260924 21:12:41.221320 14343 consensus_queue.cc:260] T 4b2681254f3242eeb2e37d2423452435 P 5ecda7f62fe647d78e4a990968bf5327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0982fc9b1f040af82515f52e5400c5f" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34739 } } peers { permanent_uuid: "cb42303b63fc4ae2a46db49707af5edb" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43821 } } peers { permanent_uuid: "5ecda7f62fe647d78e4a990968bf5327" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37515 } }
I20260924 21:12:41.221902 14343 ts_tablet_manager.cc:1428] T 4b2681254f3242eeb2e37d2423452435 P 5ecda7f62fe647d78e4a990968bf5327: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:12:41.222100 14318 heartbeater.cc:503] Master 127.6.210.190:39481 was elected leader, sending a full tablet report...
I20260924 21:12:41.223042 14345 tablet_bootstrap.cc:654] T 4b2681254f3242eeb2e37d2423452435 P cb42303b63fc4ae2a46db49707af5edb: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:41.223268 14345 log.cc:824] T 4b2681254f3242eeb2e37d2423452435 P cb42303b63fc4ae2a46db49707af5edb: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:41.223841 14345 tablet_bootstrap.cc:492] T 4b2681254f3242eeb2e37d2423452435 P cb42303b63fc4ae2a46db49707af5edb: No bootstrap required, opened a new log
I20260924 21:12:41.223892 14345 ts_tablet_manager.cc:1397] T 4b2681254f3242eeb2e37d2423452435 P cb42303b63fc4ae2a46db49707af5edb: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260924 21:12:41.225543 14345 raft_consensus.cc:348] T 4b2681254f3242eeb2e37d2423452435 P cb42303b63fc4ae2a46db49707af5edb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0982fc9b1f040af82515f52e5400c5f" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34739 } } peers { permanent_uuid: "cb42303b63fc4ae2a46db49707af5edb" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43821 } } peers { permanent_uuid: "5ecda7f62fe647d78e4a990968bf5327" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37515 } }
I20260924 21:12:41.225661 14345 raft_consensus.cc:374] T 4b2681254f3242eeb2e37d2423452435 P cb42303b63fc4ae2a46db49707af5edb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:41.225684 14345 raft_consensus.cc:729] T 4b2681254f3242eeb2e37d2423452435 P cb42303b63fc4ae2a46db49707af5edb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb42303b63fc4ae2a46db49707af5edb, State: Initialized, Role: FOLLOWER
I20260924 21:12:41.225795 14345 consensus_queue.cc:260] T 4b2681254f3242eeb2e37d2423452435 P cb42303b63fc4ae2a46db49707af5edb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0982fc9b1f040af82515f52e5400c5f" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34739 } } peers { permanent_uuid: "cb42303b63fc4ae2a46db49707af5edb" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43821 } } peers { permanent_uuid: "5ecda7f62fe647d78e4a990968bf5327" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37515 } }
I20260924 21:12:41.226051 14045 heartbeater.cc:503] Master 127.6.210.190:39481 was elected leader, sending a full tablet report...
I20260924 21:12:41.226053 14345 ts_tablet_manager.cc:1428] T 4b2681254f3242eeb2e37d2423452435 P cb42303b63fc4ae2a46db49707af5edb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:12:41.257121 14340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:41.254306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:46595 (local address 127.0.0.1:40960)
0924 21:12:41.254371 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:41.254383 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:41.254433 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:41.254788 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:41.254789 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:41.254791 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:41.254846 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:41.254848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.255506 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.255507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:41.256268 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:41.256270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.256871 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.256872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:41.256928 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:41.256931 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:12:41.256974 (+ 43us) client_negotiation.cc:240] Negotiation successful
0924 21:12:41.257022 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260924 21:12:41.257097 14328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:41.254468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40960 (local address 127.6.210.188:46595)
0924 21:12:41.254541 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:41.254543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:41.254550 (+ 7us) server_negotiation.cc:407] Connection header received
0924 21:12:41.254576 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:41.254577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:41.254596 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:41.254783 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:41.254924 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.255419 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.256366 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.256776 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.256873 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:41.256960 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:41.256982 (+ 22us) server_negotiation.cc:299] Negotiation successful
0924 21:12:41.257020 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260924 21:12:41.257570 13870 raft_consensus.cc:1203] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260924 21:12:41.258549 13904 sys_catalog.cc:455] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34863bba7e8146cab737678d7499c335. Latest consensus state: current_term: 1 leader_uuid: "34863bba7e8146cab737678d7499c335" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: NON_VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: true } } }
I20260924 21:12:41.258661 13904 sys_catalog.cc:458] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [sys.catalog]: This master's current role is: LEARNER
W20260924 21:12:41.297458 14184 tablet.cc:2369] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 21:12:41.418354 14047 tablet.cc:2369] T 4b2681254f3242eeb2e37d2423452435 P cb42303b63fc4ae2a46db49707af5edb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 21:12:41.436471 14320 tablet.cc:2369] T 4b2681254f3242eeb2e37d2423452435 P 5ecda7f62fe647d78e4a990968bf5327: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:12:41.486732 14348 raft_consensus.cc:482] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:12:41.486842 14348 raft_consensus.cc:504] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0982fc9b1f040af82515f52e5400c5f" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34739 } } peers { permanent_uuid: "cb42303b63fc4ae2a46db49707af5edb" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43821 } } peers { permanent_uuid: "5ecda7f62fe647d78e4a990968bf5327" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37515 } }
I20260924 21:12:41.487246 14348 leader_election.cc:288] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb42303b63fc4ae2a46db49707af5edb (127.6.210.129:43821), 5ecda7f62fe647d78e4a990968bf5327 (127.6.210.131:37515)
I20260924 21:12:41.490233 14187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:41.487460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:43821 (local address 127.6.210.130:41179)
0924 21:12:41.487520 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:41.487530 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:41.487573 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:41.487814 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:41.487815 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:41.487816 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:41.487897 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:41.487898 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.488530 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.488531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:41.489443 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:41.489462 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.489989 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.489990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:41.490063 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:41.490066 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:12:41.490117 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 21:12:41.490145 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":64}
I20260924 21:12:41.490284 14341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:41.487619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:41179 (local address 127.6.210.129:43821)
0924 21:12:41.487693 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:41.487695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:41.487702 (+ 7us) server_negotiation.cc:407] Connection header received
0924 21:12:41.487729 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:41.487730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:41.487740 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:41.487790 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:41.487975 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.488447 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.489554 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.489905 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.489963 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:41.489992 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:41.490146 (+ 154us) server_negotiation.cc:299] Negotiation successful
0924 21:12:41.490191 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260924 21:12:41.490362 14186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:41.487448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:37515 (local address 127.6.210.130:60159)
0924 21:12:41.487517 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:41.487530 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:41.487573 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:41.487752 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:41.487753 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:41.487755 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:41.487852 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:41.487854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.488574 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.488575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:41.489411 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:41.489413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.490025 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.490026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:41.490095 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:41.490097 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 21:12:41.490202 (+ 105us) client_negotiation.cc:240] Negotiation successful
0924 21:12:41.490238 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":1}
I20260924 21:12:41.490374 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:41.487533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:60159 (local address 127.6.210.131:37515)
0924 21:12:41.487599 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:41.487602 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:41.487616 (+ 14us) server_negotiation.cc:407] Connection header received
0924 21:12:41.487646 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:41.487647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:41.487657 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:41.487743 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:41.487939 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.488461 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.489491 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.489914 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.490007 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:41.490015 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:41.490223 (+ 208us) server_negotiation.cc:299] Negotiation successful
0924 21:12:41.490306 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 21:12:41.490672 14271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b2681254f3242eeb2e37d2423452435" candidate_uuid: "d0982fc9b1f040af82515f52e5400c5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ecda7f62fe647d78e4a990968bf5327" is_pre_election: true
I20260924 21:12:41.490690 13998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b2681254f3242eeb2e37d2423452435" candidate_uuid: "d0982fc9b1f040af82515f52e5400c5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb42303b63fc4ae2a46db49707af5edb" is_pre_election: true
I20260924 21:12:41.490832 13998 raft_consensus.cc:2454] T 4b2681254f3242eeb2e37d2423452435 P cb42303b63fc4ae2a46db49707af5edb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0982fc9b1f040af82515f52e5400c5f in term 0.
I20260924 21:12:41.490839 14271 raft_consensus.cc:2454] T 4b2681254f3242eeb2e37d2423452435 P 5ecda7f62fe647d78e4a990968bf5327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0982fc9b1f040af82515f52e5400c5f in term 0.
I20260924 21:12:41.491070 14070 leader_election.cc:302] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb42303b63fc4ae2a46db49707af5edb, d0982fc9b1f040af82515f52e5400c5f; no voters:
I20260924 21:12:41.491223 14348 raft_consensus.cc:2781] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:12:41.491276 14348 raft_consensus.cc:482] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:12:41.491302 14348 raft_consensus.cc:3037] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:41.491999 14348 raft_consensus.cc:504] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0982fc9b1f040af82515f52e5400c5f" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34739 } } peers { permanent_uuid: "cb42303b63fc4ae2a46db49707af5edb" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43821 } } peers { permanent_uuid: "5ecda7f62fe647d78e4a990968bf5327" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37515 } }
I20260924 21:12:41.492165 14348 leader_election.cc:288] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f [CANDIDATE]: Term 1 election: Requested vote from peers cb42303b63fc4ae2a46db49707af5edb (127.6.210.129:43821), 5ecda7f62fe647d78e4a990968bf5327 (127.6.210.131:37515)
I20260924 21:12:41.492306 13998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b2681254f3242eeb2e37d2423452435" candidate_uuid: "d0982fc9b1f040af82515f52e5400c5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb42303b63fc4ae2a46db49707af5edb"
I20260924 21:12:41.492350 14271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b2681254f3242eeb2e37d2423452435" candidate_uuid: "d0982fc9b1f040af82515f52e5400c5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ecda7f62fe647d78e4a990968bf5327"
I20260924 21:12:41.492394 13998 raft_consensus.cc:3037] T 4b2681254f3242eeb2e37d2423452435 P cb42303b63fc4ae2a46db49707af5edb [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:41.492429 14271 raft_consensus.cc:3037] T 4b2681254f3242eeb2e37d2423452435 P 5ecda7f62fe647d78e4a990968bf5327 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:41.493251 14271 raft_consensus.cc:2454] T 4b2681254f3242eeb2e37d2423452435 P 5ecda7f62fe647d78e4a990968bf5327 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0982fc9b1f040af82515f52e5400c5f in term 1.
I20260924 21:12:41.493251 13998 raft_consensus.cc:2454] T 4b2681254f3242eeb2e37d2423452435 P cb42303b63fc4ae2a46db49707af5edb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0982fc9b1f040af82515f52e5400c5f in term 1.
I20260924 21:12:41.493417 14069 leader_election.cc:302] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ecda7f62fe647d78e4a990968bf5327, d0982fc9b1f040af82515f52e5400c5f; no voters:
I20260924 21:12:41.493501 14348 raft_consensus.cc:2781] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:41.493685 14348 raft_consensus.cc:686] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f [term 1 LEADER]: Becoming Leader. State: Replica: d0982fc9b1f040af82515f52e5400c5f, State: Running, Role: LEADER
I20260924 21:12:41.493786 14348 consensus_queue.cc:237] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0982fc9b1f040af82515f52e5400c5f" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34739 } } peers { permanent_uuid: "cb42303b63fc4ae2a46db49707af5edb" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43821 } } peers { permanent_uuid: "5ecda7f62fe647d78e4a990968bf5327" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37515 } }
I20260924 21:12:41.494665 13081 catalog_manager.cc:5495] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f reported cstate change: term changed from 0 to 1, leader changed from <none> to d0982fc9b1f040af82515f52e5400c5f (127.6.210.130). New cstate: current_term: 1 leader_uuid: "d0982fc9b1f040af82515f52e5400c5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0982fc9b1f040af82515f52e5400c5f" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34739 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb42303b63fc4ae2a46db49707af5edb" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ecda7f62fe647d78e4a990968bf5327" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37515 } health_report { overall_health: UNKNOWN } } }
I20260924 21:12:41.495306 14357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "34863bba7e8146cab737678d7499c335" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: NON_VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: true } } }
I20260924 21:12:41.495409 14357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:41.496029 13640 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335: attempting to promote NON_VOTER faabfe445f404446a8f8eef00b4b6037 to VOTER
I20260924 21:12:41.496152 13640 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: false } }
I20260924 21:12:41.496523 13870 raft_consensus.cc:1261] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [term 1 LEARNER]: Refusing update from remote peer 34863bba7e8146cab737678d7499c335: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260924 21:12:41.496539 13690 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [term 1 FOLLOWER]: Refusing update from remote peer 34863bba7e8146cab737678d7499c335: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260924 21:12:41.496692 14357 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260924 21:12:41.496794 13642 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260924 21:12:41.497437 14357 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, faabfe445f404446a8f8eef00b4b6037 (127.6.210.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: false } } }
I20260924 21:12:41.497560 13690 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, faabfe445f404446a8f8eef00b4b6037 (127.6.210.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: false } } }
I20260924 21:12:41.497658 13870 raft_consensus.cc:2932] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, faabfe445f404446a8f8eef00b4b6037 (127.6.210.188) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: false } } }
I20260924 21:12:41.498373 14357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "34863bba7e8146cab737678d7499c335" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: false } } }
I20260924 21:12:41.498453 14357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:41.498523 13640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "34863bba7e8146cab737678d7499c335" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: false } } }
I20260924 21:12:41.498548 13642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "34863bba7e8146cab737678d7499c335" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: false } } }
I20260924 21:12:41.498600 13640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:41.498612 13642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:41.498682 14359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "34863bba7e8146cab737678d7499c335" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: false } } }
I20260924 21:12:41.498772 14359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:12:41.498728 13904 sys_catalog.cc:455] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "34863bba7e8146cab737678d7499c335" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: false } } }
I20260924 21:12:41.498806 13904 sys_catalog.cc:458] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:12:41.568125 14339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:41.565633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58832 (local address 127.6.210.130:34739)
0924 21:12:41.565703 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:41.565705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:41.565714 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:12:41.565816 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:41.565817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:41.565844 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:41.565923 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:12:41.566065 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.566578 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.567334 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.567531 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.567608 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:41.567793 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:12:41.567896 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:12:41.567994 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:41.568012 (+ 18us) server_negotiation.cc:299] Negotiation successful
0924 21:12:41.568031 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260924 21:12:41.570004 13998 raft_consensus.cc:1261] T 4b2681254f3242eeb2e37d2423452435 P cb42303b63fc4ae2a46db49707af5edb [term 1 FOLLOWER]: Refusing update from remote peer d0982fc9b1f040af82515f52e5400c5f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:41.570004 14271 raft_consensus.cc:1261] T 4b2681254f3242eeb2e37d2423452435 P 5ecda7f62fe647d78e4a990968bf5327 [term 1 FOLLOWER]: Refusing update from remote peer d0982fc9b1f040af82515f52e5400c5f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:41.570387 14348 consensus_queue.cc:1035] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ecda7f62fe647d78e4a990968bf5327" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:12:41.570552 14355 consensus_queue.cc:1035] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb42303b63fc4ae2a46db49707af5edb" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:12:41.573817 14361 mvcc.cc:205] Tried to move back new op lower bound from 7333004744987430912 to 7333004744679174144. Current Snapshot: MvccSnapshot[applied={T|T < 7333004744987430912}]
I20260924 21:12:41.573923 14363 mvcc.cc:205] Tried to move back new op lower bound from 7333004744987430912 to 7333004744679174144. Current Snapshot: MvccSnapshot[applied={T|T < 7333004744987430912}]
I20260924 21:12:41.575793 13081 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:12:41.575872 13081 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:12:41.577489 13081 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34863bba7e8146cab737678d7499c335: Sending DeleteTablet for 3 replicas of tablet 4b2681254f3242eeb2e37d2423452435
I20260924 21:12:41.577798 13978 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b2681254f3242eeb2e37d2423452435 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:41 UTC) from {username='oryx', principal='oryx/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:35272
I20260924 21:12:41.577818 14115 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b2681254f3242eeb2e37d2423452435 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:41 UTC) from {username='oryx', principal='oryx/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:58828
I20260924 21:12:41.577989 14371 tablet_replica.cc:330] T 4b2681254f3242eeb2e37d2423452435 P cb42303b63fc4ae2a46db49707af5edb: stopping tablet replica
I20260924 21:12:41.578040 14372 tablet_replica.cc:330] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f: stopping tablet replica
I20260924 21:12:41.578027 14251 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b2681254f3242eeb2e37d2423452435 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:41 UTC) from {username='oryx', principal='oryx/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:46712
I20260924 21:12:41.578081 14371 raft_consensus.cc:2229] T 4b2681254f3242eeb2e37d2423452435 P cb42303b63fc4ae2a46db49707af5edb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:12:41.578111 14372 raft_consensus.cc:2229] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:12:41.578183 14371 raft_consensus.cc:2258] T 4b2681254f3242eeb2e37d2423452435 P cb42303b63fc4ae2a46db49707af5edb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:41.578203 14372 raft_consensus.cc:2258] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:41.578343 14373 tablet_replica.cc:330] T 4b2681254f3242eeb2e37d2423452435 P 5ecda7f62fe647d78e4a990968bf5327: stopping tablet replica
I20260924 21:12:41.578536 14373 raft_consensus.cc:2229] T 4b2681254f3242eeb2e37d2423452435 P 5ecda7f62fe647d78e4a990968bf5327 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:12:41.578629 14373 raft_consensus.cc:2258] T 4b2681254f3242eeb2e37d2423452435 P 5ecda7f62fe647d78e4a990968bf5327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:41.578620 14372 ts_tablet_manager.cc:1905] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:41.579175 14373 ts_tablet_manager.cc:1905] T 4b2681254f3242eeb2e37d2423452435 P 5ecda7f62fe647d78e4a990968bf5327: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:41.579886 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 13050
I20260924 21:12:41.580456 14372 ts_tablet_manager.cc:1918] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:12:41.580518 14372 log.cc:1196] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-1/wal/wals/4b2681254f3242eeb2e37d2423452435
I20260924 21:12:41.580766 14373 ts_tablet_manager.cc:1918] T 4b2681254f3242eeb2e37d2423452435 P 5ecda7f62fe647d78e4a990968bf5327: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:12:41.580783 14372 ts_tablet_manager.cc:1939] T 4b2681254f3242eeb2e37d2423452435 P d0982fc9b1f040af82515f52e5400c5f: Deleting consensus metadata
I20260924 21:12:41.580827 14373 log.cc:1196] T 4b2681254f3242eeb2e37d2423452435 P 5ecda7f62fe647d78e4a990968bf5327: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-2/wal/wals/4b2681254f3242eeb2e37d2423452435
I20260924 21:12:41.581072 14373 ts_tablet_manager.cc:1939] T 4b2681254f3242eeb2e37d2423452435 P 5ecda7f62fe647d78e4a990968bf5327: Deleting consensus metadata
I20260924 21:12:41.582600 14371 ts_tablet_manager.cc:1905] T 4b2681254f3242eeb2e37d2423452435 P cb42303b63fc4ae2a46db49707af5edb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:41.583876 14371 ts_tablet_manager.cc:1918] T 4b2681254f3242eeb2e37d2423452435 P cb42303b63fc4ae2a46db49707af5edb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:12:41.584014 14371 log.cc:1196] T 4b2681254f3242eeb2e37d2423452435 P cb42303b63fc4ae2a46db49707af5edb: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-0/wal/wals/4b2681254f3242eeb2e37d2423452435
I20260924 21:12:41.584348 14371 ts_tablet_manager.cc:1939] T 4b2681254f3242eeb2e37d2423452435 P cb42303b63fc4ae2a46db49707af5edb: Deleting consensus metadata
W20260924 21:12:41.586615 14204 connection.cc:535] server connection from 127.0.0.1:46712 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260924 21:12:41.586612 14069 connection.cc:535] server connection from 127.0.0.1:58828 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260924 21:12:41.586668 13932 connection.cc:535] server connection from 127.0.0.1:35272 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 21:12:41.591918 14328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:41.579026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40966 (local address 127.6.210.188:46595)
0924 21:12:41.579064 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:41.579066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:41.579074 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:12:41.579093 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:41.579094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:41.579103 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:41.579159 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:41.584456 (+ 5297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.585106 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.585971 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.586295 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.586356 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:41.588445 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:41.588447 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:41.588448 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:41.588456 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:41.590281 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:41.590893 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:41.590894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:41.590894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:41.590955 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:41.591162 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:41.591163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:41.591163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:41.591293 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:41.591381 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:41.591724 (+ 343us) server_negotiation.cc:299] Negotiation successful
0924 21:12:41.591816 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 21:12:41.596177 13718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:41.588606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58772 (local address 127.6.210.189:44341)
0924 21:12:41.588649 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:41.588651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:41.588883 (+ 232us) server_negotiation.cc:407] Connection header received
0924 21:12:41.588987 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:41.588988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:41.589001 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:41.589035 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:41.589751 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.590513 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.591879 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.592177 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.592218 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:41.593482 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:41.593483 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:41.593484 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:41.593490 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:41.594995 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:41.595346 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:41.595347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:41.595348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:41.595401 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:41.595649 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:41.595651 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:41.595651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:41.595802 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:41.595931 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:41.596012 (+ 81us) server_negotiation.cc:299] Negotiation successful
0924 21:12:41.596084 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 21:12:41.596740 14050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:41.588269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40974 (local address 127.6.210.188:46595)
0924 21:12:41.588310 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:41.588312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:41.588547 (+ 235us) server_negotiation.cc:407] Connection header received
0924 21:12:41.588654 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:41.588654 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:41.588664 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:41.588694 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:41.589355 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.590098 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.591345 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:41.591645 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:41.591687 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:41.593569 (+ 1882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:41.593570 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:41.593570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:41.593577 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:41.595261 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:41.595812 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:41.595813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:41.595814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:41.595868 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:41.596068 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:41.596069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:41.596070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:41.596186 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:41.596282 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:41.596549 (+ 267us) server_negotiation.cc:299] Negotiation successful
0924 21:12:41.596623 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 21:12:42.574457 14382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:42.574140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.6.210.131:60823)
0924 21:12:42.574308 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:42.574398 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:39481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":71,"threads_started":1}
W20260924 21:12:42.574787 14318 heartbeater.cc:644] Failed to heartbeat to 127.6.210.190:39481 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.210.190:39481: connect: Connection refused (error 111)
I20260924 21:12:42.575300 14383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:42.575019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.6.210.130:33495)
0924 21:12:42.575197 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:42.575258 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:39481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":48,"threads_started":1}
I20260924 21:12:42.575403 14384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:42.575175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.6.210.129:51099)
0924 21:12:42.575288 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:42.575333 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:39481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":48,"threads_started":1}
W20260924 21:12:42.575480 14182 heartbeater.cc:644] Failed to heartbeat to 127.6.210.190:39481 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.210.190:39481: connect: Connection refused (error 111)
W20260924 21:12:42.575572 14045 heartbeater.cc:644] Failed to heartbeat to 127.6.210.190:39481 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.210.190:39481: connect: Connection refused (error 111)
I20260924 21:12:43.077857 14385 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34863bba7e8146cab737678d7499c335)
I20260924 21:12:43.077978 14385 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: false } }
I20260924 21:12:43.078442 14385 leader_election.cc:288] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34863bba7e8146cab737678d7499c335 (127.6.210.190:39481), faabfe445f404446a8f8eef00b4b6037 (127.6.210.188:46595)
I20260924 21:12:43.078445 14386 raft_consensus.cc:482] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 34863bba7e8146cab737678d7499c335)
I20260924 21:12:43.078634 14386 raft_consensus.cc:504] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: false } }
I20260924 21:12:43.078745 14387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.078541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.0.0.1:57482)
0924 21:12:43.078657 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:43.078705 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:39481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":61,"threads_started":1}
W20260924 21:12:43.078852 13666 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.210.190:39481: connect: Connection refused (error 111)
I20260924 21:12:43.078881 14386 leader_election.cc:288] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 34863bba7e8146cab737678d7499c335 (127.6.210.190:39481), 9bbea6a1f4d844e9a0dedfbaa1c60776 (127.6.210.189:44341)
I20260924 21:12:43.079312 14387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.079179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.0.0.1:57490)
0924 21:12:43.079248 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:43.079281 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:39481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260924 21:12:43.079494 14392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.079024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.0.0.1:57488)
0924 21:12:43.079398 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:43.079450 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:39481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":40,"threads_started":1}
W20260924 21:12:43.079505 13666 leader_election.cc:334] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 34863bba7e8146cab737678d7499c335 (127.6.210.190:39481): Network error: Client connection negotiation failed: client connection to 127.6.210.190:39481: connect: Connection refused (error 111)
W20260924 21:12:43.079644 13851 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.210.190:39481: connect: Connection refused (error 111)
I20260924 21:12:43.080099 14392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.079988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.0.0.1:57498)
0924 21:12:43.080040 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:43.080076 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:39481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260924 21:12:43.080174 13851 leader_election.cc:334] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 34863bba7e8146cab737678d7499c335 (127.6.210.190:39481): Network error: Client connection negotiation failed: client connection to 127.6.210.190:39481: connect: Connection refused (error 111)
I20260924 21:12:43.082173 14389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.078749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40986 (local address 127.6.210.188:46595)
0924 21:12:43.078842 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:43.078844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:43.079021 (+ 177us) server_negotiation.cc:407] Connection header received
0924 21:12:43.079049 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:43.079050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:43.079062 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:43.079318 (+ 256us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:43.079521 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:43.080258 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:43.081321 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:43.081841 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:43.081891 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:43.081899 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:43.082063 (+ 164us) server_negotiation.cc:299] Negotiation successful
0924 21:12:43.082101 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71,"thread_start_us":44,"threads_started":1}
I20260924 21:12:43.082185 14388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.078603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:46595 (local address 127.0.0.1:40986)
0924 21:12:43.078892 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:43.078901 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:43.078962 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:43.079377 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:43.079378 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:43.079381 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:43.079437 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:43.079440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:43.080325 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:43.080326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:43.081238 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:43.081240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:43.081907 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:43.081908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:43.081969 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:43.081973 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:12:43.082029 (+ 56us) client_negotiation.cc:240] Negotiation successful
0924 21:12:43.082117 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":54,"threads_started":1}
I20260924 21:12:43.082517 13870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "faabfe445f404446a8f8eef00b4b6037" is_pre_election: true
I20260924 21:12:43.082616 14393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.079185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58788 (local address 127.6.210.189:44341)
0924 21:12:43.079328 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:43.079330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:43.079335 (+ 5us) server_negotiation.cc:407] Connection header received
0924 21:12:43.079352 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:43.079352 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:43.079383 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:43.079919 (+ 536us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:43.079954 (+ 35us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:43.080688 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:43.081755 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:43.082245 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:43.082363 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:43.082373 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:43.082506 (+ 133us) server_negotiation.cc:299] Negotiation successful
0924 21:12:43.082558 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":109,"threads_started":1}
I20260924 21:12:43.082623 14391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.078987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:44341 (local address 127.0.0.1:58788)
0924 21:12:43.079159 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:43.079169 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:43.079221 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:43.079475 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:43.079476 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:43.079479 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:43.079534 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:43.079536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:43.080773 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:43.080774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:43.081677 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:43.081679 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:43.082369 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:43.082370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:43.082434 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:43.082437 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:12:43.082485 (+ 48us) client_negotiation.cc:240] Negotiation successful
0924 21:12:43.082527 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":45,"threads_started":1}
I20260924 21:12:43.082652 13870 raft_consensus.cc:2454] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9bbea6a1f4d844e9a0dedfbaa1c60776 in term 1.
I20260924 21:12:43.082865 13667 leader_election.cc:302] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9bbea6a1f4d844e9a0dedfbaa1c60776, faabfe445f404446a8f8eef00b4b6037; no voters: 34863bba7e8146cab737678d7499c335
I20260924 21:12:43.082922 13690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "faabfe445f404446a8f8eef00b4b6037" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" is_pre_election: true
I20260924 21:12:43.082968 14385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 21:12:43.083021 14385 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 34863bba7e8146cab737678d7499c335)
I20260924 21:12:43.083040 14385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [term 1 FOLLOWER]: Advancing to term 2
I20260924 21:12:43.083686 14385 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: false } }
I20260924 21:12:43.083810 13690 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate faabfe445f404446a8f8eef00b4b6037 in current term 2: Already voted for candidate 9bbea6a1f4d844e9a0dedfbaa1c60776 in this term.
I20260924 21:12:43.083830 14385 leader_election.cc:288] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [CANDIDATE]: Term 2 election: Requested vote from peers 34863bba7e8146cab737678d7499c335 (127.6.210.190:39481), faabfe445f404446a8f8eef00b4b6037 (127.6.210.188:46595)
I20260924 21:12:43.083973 14388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.083896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.0.0.1:57504)
0924 21:12:43.083925 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:43.083951 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:39481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260924 21:12:43.083950 13870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "faabfe445f404446a8f8eef00b4b6037"
I20260924 21:12:43.083999 13850 leader_election.cc:302] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: faabfe445f404446a8f8eef00b4b6037; no voters: 34863bba7e8146cab737678d7499c335, 9bbea6a1f4d844e9a0dedfbaa1c60776
I20260924 21:12:43.084013 13870 raft_consensus.cc:3037] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [term 1 FOLLOWER]: Advancing to term 2
I20260924 21:12:43.084316 14388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.084226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.0.0.1:57506)
0924 21:12:43.084272 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:43.084295 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:39481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260924 21:12:43.084424 13666 leader_election.cc:334] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 34863bba7e8146cab737678d7499c335 (127.6.210.190:39481): Network error: Client connection negotiation failed: client connection to 127.6.210.190:39481: connect: Connection refused (error 111)
I20260924 21:12:43.084713 13870 raft_consensus.cc:2454] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bbea6a1f4d844e9a0dedfbaa1c60776 in term 2.
I20260924 21:12:43.084790 14386 raft_consensus.cc:2726] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 21:12:43.084872 13667 leader_election.cc:302] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9bbea6a1f4d844e9a0dedfbaa1c60776, faabfe445f404446a8f8eef00b4b6037; no voters: 34863bba7e8146cab737678d7499c335
I20260924 21:12:43.084966 14385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 21:12:43.085069 14385 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [term 2 LEADER]: Becoming Leader. State: Replica: 9bbea6a1f4d844e9a0dedfbaa1c60776, State: Running, Role: LEADER
I20260924 21:12:43.085124 14385 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: false } }
I20260924 21:12:43.085462 14395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9bbea6a1f4d844e9a0dedfbaa1c60776. Latest consensus state: current_term: 2 leader_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: false } } }
I20260924 21:12:43.085597 14395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:43.085772 14397 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:12:43.086113 14397 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:12:43.086167 14397 catalog_manager.cc:1259] Loaded cluster ID: 91a4d8d24fc44b61b0e4d153ce600838
I20260924 21:12:43.086184 14397 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:12:43.086360 14397 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:12:43.086426 14397 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776: Loaded TSK: 0
I20260924 21:12:43.086678 14397 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:12:43.129415 13680 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58772:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:12:43.131016 13680 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:12:43.132225 13870 raft_consensus.cc:1261] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [term 2 FOLLOWER]: Refusing update from remote peer 9bbea6a1f4d844e9a0dedfbaa1c60776: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260924 21:12:43.132244 14388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.132077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.0.0.1:57514)
0924 21:12:43.132174 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:43.132210 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:39481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260924 21:12:43.132567 14385 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [LEADER]: Connected to new peer: Peer: permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260924 21:12:43.132633 14388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.132544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.0.0.1:57528)
0924 21:12:43.132573 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:43.132596 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:39481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260924 21:12:43.132752 13666 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 -> Peer 34863bba7e8146cab737678d7499c335 (127.6.210.190:39481): Couldn't send request to peer 34863bba7e8146cab737678d7499c335. Status: Network error: Client connection negotiation failed: client connection to 127.6.210.190:39481: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 21:12:43.133226 14386 sys_catalog.cc:455] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9bbea6a1f4d844e9a0dedfbaa1c60776. Latest consensus state: current_term: 2 leader_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: false } } }
I20260924 21:12:43.133329 14386 sys_catalog.cc:458] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:12:43.133703 14395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: false } } }
I20260924 21:12:43.133797 14395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:43.134177 14399 mvcc.cc:205] Tried to move back new op lower bound from 7333004751387160576 to 7333004751197310976. Current Snapshot: MvccSnapshot[applied={T|T < 7333004751387160576}]
I20260924 21:12:43.134260 14386 sys_catalog.cc:455] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "34863bba7e8146cab737678d7499c335" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39481 } } peers { permanent_uuid: "9bbea6a1f4d844e9a0dedfbaa1c60776" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "faabfe445f404446a8f8eef00b4b6037" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 46595 } attrs { promote: false } } }
I20260924 21:12:43.134363 14386 sys_catalog.cc:458] T 00000000000000000000000000000000 P faabfe445f404446a8f8eef00b4b6037 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:12:43.138790 14387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.135513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:34739 (local address 127.0.0.1:58834)
0924 21:12:43.135751 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:43.135759 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:43.135826 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:43.135991 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:43.135992 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:43.135996 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:43.136042 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:43.136043 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:43.137043 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:43.137044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:43.137829 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:43.137831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:43.138555 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:43.138556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:43.138631 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:43.138635 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:12:43.138663 (+ 28us) client_negotiation.cc:240] Negotiation successful
0924 21:12:43.138702 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":47}
I20260924 21:12:43.138808 14405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.135709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35282 (local address 127.6.210.129:43821)
0924 21:12:43.135835 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:43.135837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:43.135846 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:12:43.135873 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:43.135873 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:43.135886 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:43.136114 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:43.136153 (+ 39us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:43.136694 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:43.137883 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:43.138436 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:43.138533 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:43.138546 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:43.138676 (+ 130us) server_negotiation.cc:299] Negotiation successful
0924 21:12:43.138716 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":58,"threads_started":1}
I20260924 21:12:43.138796 14388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.135512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:43821 (local address 127.0.0.1:35282)
0924 21:12:43.135564 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:43.135575 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:43.135622 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:43.135976 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:43.135978 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:43.135979 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:43.136043 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:43.136044 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:43.136783 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:43.136783 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:43.137793 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:43.137795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:43.138525 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:43.138526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:43.138584 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:43.138588 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:12:43.138668 (+ 80us) client_negotiation.cc:240] Negotiation successful
0924 21:12:43.138709 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":1}
I20260924 21:12:43.139091 14403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.135597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58834 (local address 127.6.210.130:34739)
0924 21:12:43.135827 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:43.135829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:43.135838 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:12:43.135888 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:43.135888 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:43.135899 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:43.136108 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:43.136150 (+ 42us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:43.136948 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:43.137930 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:43.138465 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:43.138911 (+ 446us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:43.138926 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:43.138968 (+ 42us) server_negotiation.cc:299] Negotiation successful
0924 21:12:43.139012 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":67,"threads_started":1}
I20260924 21:12:43.139091 13978 tablet_service.cc:1467] Processing CreateTablet for tablet ae58a1db8198450fb2e8341f615697fb (DEFAULT_TABLE table=test-table [id=1ca6a9ec81b246d3947be206ecb17cf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:43.139209 13978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae58a1db8198450fb2e8341f615697fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:43.139263 14115 tablet_service.cc:1467] Processing CreateTablet for tablet ae58a1db8198450fb2e8341f615697fb (DEFAULT_TABLE table=test-table [id=1ca6a9ec81b246d3947be206ecb17cf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:43.139331 14404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.135642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:37515 (local address 127.0.0.1:46726)
0924 21:12:43.135757 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:43.135766 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:43.135812 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:43.136188 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:43.136188 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:43.136190 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:43.136229 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:43.136230 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:43.137158 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:43.137159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:43.138213 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:43.138215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:43.139103 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:43.139104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:43.139179 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:43.139183 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:12:43.139232 (+ 49us) client_negotiation.cc:240] Negotiation successful
0924 21:12:43.139271 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":71,"threads_started":1}
I20260924 21:12:43.139372 14115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae58a1db8198450fb2e8341f615697fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:43.139539 14406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.135943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46726 (local address 127.6.210.131:37515)
0924 21:12:43.136047 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:43.136049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:43.136056 (+ 7us) server_negotiation.cc:407] Connection header received
0924 21:12:43.136081 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:43.136082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:43.136091 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:43.136147 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:43.136309 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:43.137066 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:43.138329 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:43.139003 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:43.139061 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:43.139072 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:43.139410 (+ 338us) server_negotiation.cc:299] Negotiation successful
0924 21:12:43.139463 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":54,"threads_started":1}
I20260924 21:12:43.140497 14407 tablet_bootstrap.cc:492] T ae58a1db8198450fb2e8341f615697fb P cb42303b63fc4ae2a46db49707af5edb: Bootstrap starting.
I20260924 21:12:43.140667 14251 tablet_service.cc:1467] Processing CreateTablet for tablet ae58a1db8198450fb2e8341f615697fb (DEFAULT_TABLE table=test-table [id=1ca6a9ec81b246d3947be206ecb17cf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:43.140803 14251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae58a1db8198450fb2e8341f615697fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:43.140889 14407 tablet_bootstrap.cc:654] T ae58a1db8198450fb2e8341f615697fb P cb42303b63fc4ae2a46db49707af5edb: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:43.141675 14407 tablet_bootstrap.cc:492] T ae58a1db8198450fb2e8341f615697fb P cb42303b63fc4ae2a46db49707af5edb: No bootstrap required, opened a new log
I20260924 21:12:43.141716 14407 ts_tablet_manager.cc:1397] T ae58a1db8198450fb2e8341f615697fb P cb42303b63fc4ae2a46db49707af5edb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:43.141880 14408 tablet_bootstrap.cc:492] T ae58a1db8198450fb2e8341f615697fb P d0982fc9b1f040af82515f52e5400c5f: Bootstrap starting.
I20260924 21:12:43.141865 14407 raft_consensus.cc:348] T ae58a1db8198450fb2e8341f615697fb P cb42303b63fc4ae2a46db49707af5edb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0982fc9b1f040af82515f52e5400c5f" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34739 } } peers { permanent_uuid: "5ecda7f62fe647d78e4a990968bf5327" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37515 } } peers { permanent_uuid: "cb42303b63fc4ae2a46db49707af5edb" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43821 } }
I20260924 21:12:43.141927 14407 raft_consensus.cc:374] T ae58a1db8198450fb2e8341f615697fb P cb42303b63fc4ae2a46db49707af5edb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:43.141942 14407 raft_consensus.cc:729] T ae58a1db8198450fb2e8341f615697fb P cb42303b63fc4ae2a46db49707af5edb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb42303b63fc4ae2a46db49707af5edb, State: Initialized, Role: FOLLOWER
I20260924 21:12:43.142025 14407 consensus_queue.cc:260] T ae58a1db8198450fb2e8341f615697fb P cb42303b63fc4ae2a46db49707af5edb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0982fc9b1f040af82515f52e5400c5f" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34739 } } peers { permanent_uuid: "5ecda7f62fe647d78e4a990968bf5327" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37515 } } peers { permanent_uuid: "cb42303b63fc4ae2a46db49707af5edb" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43821 } }
I20260924 21:12:43.142328 14408 tablet_bootstrap.cc:654] T ae58a1db8198450fb2e8341f615697fb P d0982fc9b1f040af82515f52e5400c5f: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:43.142174 14407 ts_tablet_manager.cc:1428] T ae58a1db8198450fb2e8341f615697fb P cb42303b63fc4ae2a46db49707af5edb: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 21:12:43.142802 14046 heartbeater.cc:503] Master 127.6.210.189:44341 was elected leader, sending a full tablet report...
I20260924 21:12:43.142901 14408 tablet_bootstrap.cc:492] T ae58a1db8198450fb2e8341f615697fb P d0982fc9b1f040af82515f52e5400c5f: No bootstrap required, opened a new log
I20260924 21:12:43.142956 14408 ts_tablet_manager.cc:1397] T ae58a1db8198450fb2e8341f615697fb P d0982fc9b1f040af82515f52e5400c5f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260924 21:12:43.143059 14408 raft_consensus.cc:348] T ae58a1db8198450fb2e8341f615697fb P d0982fc9b1f040af82515f52e5400c5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0982fc9b1f040af82515f52e5400c5f" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34739 } } peers { permanent_uuid: "5ecda7f62fe647d78e4a990968bf5327" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37515 } } peers { permanent_uuid: "cb42303b63fc4ae2a46db49707af5edb" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43821 } }
I20260924 21:12:43.143131 14408 raft_consensus.cc:374] T ae58a1db8198450fb2e8341f615697fb P d0982fc9b1f040af82515f52e5400c5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:43.143162 14408 raft_consensus.cc:729] T ae58a1db8198450fb2e8341f615697fb P d0982fc9b1f040af82515f52e5400c5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0982fc9b1f040af82515f52e5400c5f, State: Initialized, Role: FOLLOWER
I20260924 21:12:43.143221 14408 consensus_queue.cc:260] T ae58a1db8198450fb2e8341f615697fb P d0982fc9b1f040af82515f52e5400c5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0982fc9b1f040af82515f52e5400c5f" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34739 } } peers { permanent_uuid: "5ecda7f62fe647d78e4a990968bf5327" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37515 } } peers { permanent_uuid: "cb42303b63fc4ae2a46db49707af5edb" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43821 } }
I20260924 21:12:43.143057 14410 tablet_bootstrap.cc:492] T ae58a1db8198450fb2e8341f615697fb P 5ecda7f62fe647d78e4a990968bf5327: Bootstrap starting.
I20260924 21:12:43.143363 14408 ts_tablet_manager.cc:1428] T ae58a1db8198450fb2e8341f615697fb P d0982fc9b1f040af82515f52e5400c5f: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260924 21:12:43.143606 14410 tablet_bootstrap.cc:654] T ae58a1db8198450fb2e8341f615697fb P 5ecda7f62fe647d78e4a990968bf5327: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:43.143901 14183 heartbeater.cc:503] Master 127.6.210.189:44341 was elected leader, sending a full tablet report...
I20260924 21:12:43.142733 14412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.142364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.6.210.129:34125)
0924 21:12:43.142642 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:43.142681 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:39481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":89,"threads_started":1}
I20260924 21:12:43.144176 14410 tablet_bootstrap.cc:492] T ae58a1db8198450fb2e8341f615697fb P 5ecda7f62fe647d78e4a990968bf5327: No bootstrap required, opened a new log
I20260924 21:12:43.144224 14410 ts_tablet_manager.cc:1397] T ae58a1db8198450fb2e8341f615697fb P 5ecda7f62fe647d78e4a990968bf5327: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:43.144330 14410 raft_consensus.cc:348] T ae58a1db8198450fb2e8341f615697fb P 5ecda7f62fe647d78e4a990968bf5327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0982fc9b1f040af82515f52e5400c5f" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34739 } } peers { permanent_uuid: "5ecda7f62fe647d78e4a990968bf5327" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37515 } } peers { permanent_uuid: "cb42303b63fc4ae2a46db49707af5edb" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43821 } }
I20260924 21:12:43.144397 14410 raft_consensus.cc:374] T ae58a1db8198450fb2e8341f615697fb P 5ecda7f62fe647d78e4a990968bf5327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:43.144424 14410 raft_consensus.cc:729] T ae58a1db8198450fb2e8341f615697fb P 5ecda7f62fe647d78e4a990968bf5327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ecda7f62fe647d78e4a990968bf5327, State: Initialized, Role: FOLLOWER
I20260924 21:12:43.144465 14412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.144214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.6.210.129:47521)
0924 21:12:43.144407 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:43.144437 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:39481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260924 21:12:43.144466 14410 consensus_queue.cc:260] T ae58a1db8198450fb2e8341f615697fb P 5ecda7f62fe647d78e4a990968bf5327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0982fc9b1f040af82515f52e5400c5f" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34739 } } peers { permanent_uuid: "5ecda7f62fe647d78e4a990968bf5327" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37515 } } peers { permanent_uuid: "cb42303b63fc4ae2a46db49707af5edb" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43821 } }
I20260924 21:12:43.144570 14415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.143554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.6.210.130:53249)
0924 21:12:43.144180 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:43.144219 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:39481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":72,"threads_started":1}
I20260924 21:12:43.144605 14410 ts_tablet_manager.cc:1428] T ae58a1db8198450fb2e8341f615697fb P 5ecda7f62fe647d78e4a990968bf5327: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 21:12:43.144924 14415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.144758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.6.210.130:37181)
0924 21:12:43.144867 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:43.144890 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:39481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":71,"threads_started":1}
I20260924 21:12:43.145026 14319 heartbeater.cc:503] Master 127.6.210.189:44341 was elected leader, sending a full tablet report...
I20260924 21:12:43.145147 14419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.144858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.6.210.131:45593)
0924 21:12:43.145075 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:43.145110 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:39481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":63,"threads_started":1}
I20260924 21:12:43.321214 14411 raft_consensus.cc:482] T ae58a1db8198450fb2e8341f615697fb P cb42303b63fc4ae2a46db49707af5edb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:12:43.321316 14411 raft_consensus.cc:504] T ae58a1db8198450fb2e8341f615697fb P cb42303b63fc4ae2a46db49707af5edb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0982fc9b1f040af82515f52e5400c5f" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34739 } } peers { permanent_uuid: "5ecda7f62fe647d78e4a990968bf5327" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37515 } } peers { permanent_uuid: "cb42303b63fc4ae2a46db49707af5edb" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43821 } }
I20260924 21:12:43.321704 14411 leader_election.cc:288] T ae58a1db8198450fb2e8341f615697fb P cb42303b63fc4ae2a46db49707af5edb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0982fc9b1f040af82515f52e5400c5f (127.6.210.130:34739), 5ecda7f62fe647d78e4a990968bf5327 (127.6.210.131:37515)
I20260924 21:12:43.324636 14403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.321955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:57703 (local address 127.6.210.130:34739)
0924 21:12:43.322001 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:43.322004 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:43.322016 (+ 12us) server_negotiation.cc:407] Connection header received
0924 21:12:43.322070 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:43.322071 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:43.322082 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:43.322152 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:43.322356 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:43.322839 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:43.323845 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:43.324262 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:43.324360 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:43.324372 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:43.324522 (+ 150us) server_negotiation.cc:299] Negotiation successful
0924 21:12:43.324561 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 21:12:43.324636 14412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.321860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:34739 (local address 127.6.210.129:57703)
0924 21:12:43.321941 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:43.321954 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:43.322011 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:43.322160 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:43.322161 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:43.322165 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:43.322240 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:43.322242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:43.322926 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:43.322927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:43.323747 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:43.323749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:43.324363 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:43.324364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:43.324438 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:43.324448 (+ 10us) client_negotiation.cc:769] Sending connection context
0924 21:12:43.324515 (+ 67us) client_negotiation.cc:240] Negotiation successful
0924 21:12:43.324563 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260924 21:12:43.324762 14420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.322009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:37515 (local address 127.6.210.129:33625)
0924 21:12:43.322234 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:43.322241 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:43.322337 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:43.322474 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:43.322475 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:43.322476 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:43.322541 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:43.322543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:43.323201 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:43.323202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:43.323920 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:43.323922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:43.324481 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:43.324482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:43.324562 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:43.324566 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:12:43.324635 (+ 69us) client_negotiation.cc:240] Negotiation successful
0924 21:12:43.324675 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":140,"threads_started":1}
I20260924 21:12:43.324750 14406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.322100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:33625 (local address 127.6.210.131:37515)
0924 21:12:43.322170 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:43.322172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:43.322314 (+ 142us) server_negotiation.cc:407] Connection header received
0924 21:12:43.322394 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:43.322395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:43.322404 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:43.322458 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:43.322620 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:43.323104 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:43.324018 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:43.324373 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:43.324459 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:43.324479 (+ 20us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:43.324642 (+ 163us) server_negotiation.cc:299] Negotiation successful
0924 21:12:43.324680 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 21:12:43.324957 14135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae58a1db8198450fb2e8341f615697fb" candidate_uuid: "cb42303b63fc4ae2a46db49707af5edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0982fc9b1f040af82515f52e5400c5f" is_pre_election: true
I20260924 21:12:43.325059 14271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae58a1db8198450fb2e8341f615697fb" candidate_uuid: "cb42303b63fc4ae2a46db49707af5edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ecda7f62fe647d78e4a990968bf5327" is_pre_election: true
I20260924 21:12:43.325140 14271 raft_consensus.cc:2454] T ae58a1db8198450fb2e8341f615697fb P 5ecda7f62fe647d78e4a990968bf5327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb42303b63fc4ae2a46db49707af5edb in term 0.
I20260924 21:12:43.325140 14135 raft_consensus.cc:2454] T ae58a1db8198450fb2e8341f615697fb P d0982fc9b1f040af82515f52e5400c5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb42303b63fc4ae2a46db49707af5edb in term 0.
I20260924 21:12:43.325297 13932 leader_election.cc:302] T ae58a1db8198450fb2e8341f615697fb P cb42303b63fc4ae2a46db49707af5edb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb42303b63fc4ae2a46db49707af5edb, d0982fc9b1f040af82515f52e5400c5f; no voters:
I20260924 21:12:43.325470 14411 raft_consensus.cc:2781] T ae58a1db8198450fb2e8341f615697fb P cb42303b63fc4ae2a46db49707af5edb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:12:43.325516 14411 raft_consensus.cc:482] T ae58a1db8198450fb2e8341f615697fb P cb42303b63fc4ae2a46db49707af5edb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:12:43.325542 14411 raft_consensus.cc:3037] T ae58a1db8198450fb2e8341f615697fb P cb42303b63fc4ae2a46db49707af5edb [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:43.326253 14411 raft_consensus.cc:504] T ae58a1db8198450fb2e8341f615697fb P cb42303b63fc4ae2a46db49707af5edb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0982fc9b1f040af82515f52e5400c5f" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34739 } } peers { permanent_uuid: "5ecda7f62fe647d78e4a990968bf5327" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37515 } } peers { permanent_uuid: "cb42303b63fc4ae2a46db49707af5edb" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43821 } }
I20260924 21:12:43.326432 14411 leader_election.cc:288] T ae58a1db8198450fb2e8341f615697fb P cb42303b63fc4ae2a46db49707af5edb [CANDIDATE]: Term 1 election: Requested vote from peers d0982fc9b1f040af82515f52e5400c5f (127.6.210.130:34739), 5ecda7f62fe647d78e4a990968bf5327 (127.6.210.131:37515)
I20260924 21:12:43.326602 14135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae58a1db8198450fb2e8341f615697fb" candidate_uuid: "cb42303b63fc4ae2a46db49707af5edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0982fc9b1f040af82515f52e5400c5f"
I20260924 21:12:43.326673 14271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae58a1db8198450fb2e8341f615697fb" candidate_uuid: "cb42303b63fc4ae2a46db49707af5edb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ecda7f62fe647d78e4a990968bf5327"
I20260924 21:12:43.326725 14135 raft_consensus.cc:3037] T ae58a1db8198450fb2e8341f615697fb P d0982fc9b1f040af82515f52e5400c5f [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:43.326735 14271 raft_consensus.cc:3037] T ae58a1db8198450fb2e8341f615697fb P 5ecda7f62fe647d78e4a990968bf5327 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:43.327527 14271 raft_consensus.cc:2454] T ae58a1db8198450fb2e8341f615697fb P 5ecda7f62fe647d78e4a990968bf5327 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb42303b63fc4ae2a46db49707af5edb in term 1.
I20260924 21:12:43.327527 14135 raft_consensus.cc:2454] T ae58a1db8198450fb2e8341f615697fb P d0982fc9b1f040af82515f52e5400c5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb42303b63fc4ae2a46db49707af5edb in term 1.
I20260924 21:12:43.327745 13932 leader_election.cc:302] T ae58a1db8198450fb2e8341f615697fb P cb42303b63fc4ae2a46db49707af5edb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb42303b63fc4ae2a46db49707af5edb, d0982fc9b1f040af82515f52e5400c5f; no voters:
I20260924 21:12:43.327845 14411 raft_consensus.cc:2781] T ae58a1db8198450fb2e8341f615697fb P cb42303b63fc4ae2a46db49707af5edb [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:43.327993 14411 raft_consensus.cc:686] T ae58a1db8198450fb2e8341f615697fb P cb42303b63fc4ae2a46db49707af5edb [term 1 LEADER]: Becoming Leader. State: Replica: cb42303b63fc4ae2a46db49707af5edb, State: Running, Role: LEADER
I20260924 21:12:43.328088 14411 consensus_queue.cc:237] T ae58a1db8198450fb2e8341f615697fb P cb42303b63fc4ae2a46db49707af5edb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0982fc9b1f040af82515f52e5400c5f" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34739 } } peers { permanent_uuid: "5ecda7f62fe647d78e4a990968bf5327" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37515 } } peers { permanent_uuid: "cb42303b63fc4ae2a46db49707af5edb" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43821 } }
I20260924 21:12:43.328951 14420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.328371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.6.210.129:34615)
0924 21:12:43.328890 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:43.328924 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:39481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":501}
I20260924 21:12:43.328848 13680 catalog_manager.cc:5495] T ae58a1db8198450fb2e8341f615697fb P cb42303b63fc4ae2a46db49707af5edb reported cstate change: term changed from 0 to 1, leader changed from <none> to cb42303b63fc4ae2a46db49707af5edb (127.6.210.129). New cstate: current_term: 1 leader_uuid: "cb42303b63fc4ae2a46db49707af5edb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0982fc9b1f040af82515f52e5400c5f" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ecda7f62fe647d78e4a990968bf5327" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb42303b63fc4ae2a46db49707af5edb" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43821 } health_report { overall_health: HEALTHY } } }
I20260924 21:12:43.365691 14405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.363288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35288 (local address 127.6.210.129:43821)
0924 21:12:43.363372 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:43.363374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:43.363383 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:12:43.363416 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:43.363417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:43.363436 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:43.363508 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:12:43.363632 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:43.364226 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:43.364937 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:43.365190 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:43.365244 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:43.365375 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:12:43.365473 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:12:43.365526 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:43.365567 (+ 41us) server_negotiation.cc:299] Negotiation successful
0924 21:12:43.365583 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260924 21:12:43.366817 14271 raft_consensus.cc:1261] T ae58a1db8198450fb2e8341f615697fb P 5ecda7f62fe647d78e4a990968bf5327 [term 1 FOLLOWER]: Refusing update from remote peer cb42303b63fc4ae2a46db49707af5edb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:43.367089 14135 raft_consensus.cc:1261] T ae58a1db8198450fb2e8341f615697fb P d0982fc9b1f040af82515f52e5400c5f [term 1 FOLLOWER]: Refusing update from remote peer cb42303b63fc4ae2a46db49707af5edb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:43.367112 14411 consensus_queue.cc:1035] T ae58a1db8198450fb2e8341f615697fb P cb42303b63fc4ae2a46db49707af5edb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ecda7f62fe647d78e4a990968bf5327" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 37515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:12:43.367350 14411 consensus_queue.cc:1035] T ae58a1db8198450fb2e8341f615697fb P cb42303b63fc4ae2a46db49707af5edb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0982fc9b1f040af82515f52e5400c5f" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 34739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:12:43.367811 14419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.367709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.6.210.131:55109)
0924 21:12:43.367751 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:43.367782 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:39481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260924 21:12:43.368678 14425 mvcc.cc:205] Tried to move back new op lower bound from 7333004752348340224 to 7333004752192319488. Current Snapshot: MvccSnapshot[applied={T|T < 7333004752348340224}]
I20260924 21:12:43.368700 14418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.368299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.6.210.130:49541)
0924 21:12:43.368614 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:43.368659 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:39481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285}
I20260924 21:12:43.369100 14419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.368995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.6.210.131:33999)
0924 21:12:43.369048 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:43.369074 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:39481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260924 21:12:43.369325 14420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.369060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.6.210.129:40713)
0924 21:12:43.369258 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:43.369290 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:39481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181}
I20260924 21:12:43.370461 14420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.370372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.6.210.129:36083)
0924 21:12:43.370413 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:43.370437 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:39481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260924 21:12:43.372005 14418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:43.371052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39481 (local address 127.6.210.130:37539)
0924 21:12:43.371947 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:43.371977 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:39481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":880}
I20260924 21:12:43.372205 13680 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:12:43.372288 13680 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:12:43.373726 13680 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9bbea6a1f4d844e9a0dedfbaa1c60776: Sending DeleteTablet for 3 replicas of tablet ae58a1db8198450fb2e8341f615697fb
I20260924 21:12:43.374037 14251 tablet_service.cc:1514] Processing DeleteTablet for tablet ae58a1db8198450fb2e8341f615697fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:43 UTC) from {username='oryx', principal='oryx/127.6.210.189@KRBTEST.COM'} at 127.0.0.1:46726
I20260924 21:12:43.374053 14115 tablet_service.cc:1514] Processing DeleteTablet for tablet ae58a1db8198450fb2e8341f615697fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:43 UTC) from {username='oryx', principal='oryx/127.6.210.189@KRBTEST.COM'} at 127.0.0.1:58834
I20260924 21:12:43.374250 13978 tablet_service.cc:1514] Processing DeleteTablet for tablet ae58a1db8198450fb2e8341f615697fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:43 UTC) from {username='oryx', principal='oryx/127.6.210.189@KRBTEST.COM'} at 127.0.0.1:35282
I20260924 21:12:43.374465 14434 tablet_replica.cc:330] T ae58a1db8198450fb2e8341f615697fb P 5ecda7f62fe647d78e4a990968bf5327: stopping tablet replica
I20260924 21:12:43.374536 14434 raft_consensus.cc:2229] T ae58a1db8198450fb2e8341f615697fb P 5ecda7f62fe647d78e4a990968bf5327 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:12:43.374501 14436 tablet_replica.cc:330] T ae58a1db8198450fb2e8341f615697fb P cb42303b63fc4ae2a46db49707af5edb: stopping tablet replica
I20260924 21:12:43.374547 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 13827
I20260924 21:12:43.374598 14434 raft_consensus.cc:2258] T ae58a1db8198450fb2e8341f615697fb P 5ecda7f62fe647d78e4a990968bf5327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:43.375952 14435 tablet_replica.cc:330] T ae58a1db8198450fb2e8341f615697fb P d0982fc9b1f040af82515f52e5400c5f: stopping tablet replica
I20260924 21:12:43.376013 14435 raft_consensus.cc:2229] T ae58a1db8198450fb2e8341f615697fb P d0982fc9b1f040af82515f52e5400c5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:12:43.376061 14435 raft_consensus.cc:2258] T ae58a1db8198450fb2e8341f615697fb P d0982fc9b1f040af82515f52e5400c5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:43.376353 14434 ts_tablet_manager.cc:1905] T ae58a1db8198450fb2e8341f615697fb P 5ecda7f62fe647d78e4a990968bf5327: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:43.376446 14435 ts_tablet_manager.cc:1905] T ae58a1db8198450fb2e8341f615697fb P d0982fc9b1f040af82515f52e5400c5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:43.378013 14435 ts_tablet_manager.cc:1918] T ae58a1db8198450fb2e8341f615697fb P d0982fc9b1f040af82515f52e5400c5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:12:43.378099 14435 log.cc:1196] T ae58a1db8198450fb2e8341f615697fb P d0982fc9b1f040af82515f52e5400c5f: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-1/wal/wals/ae58a1db8198450fb2e8341f615697fb
I20260924 21:12:43.378302 14434 ts_tablet_manager.cc:1918] T ae58a1db8198450fb2e8341f615697fb P 5ecda7f62fe647d78e4a990968bf5327: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:12:43.378372 14434 log.cc:1196] T ae58a1db8198450fb2e8341f615697fb P 5ecda7f62fe647d78e4a990968bf5327: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284333163253-6986-0/minicluster-data/ts-2/wal/wals/ae58a1db8198450fb2e8341f615697fb
I20260924 21:12:43.378453 14435 ts_tablet_manager.cc:1939] T ae58a1db8198450fb2e8341f615697fb P d0982fc9b1f040af82515f52e5400c5f: Deleting consensus metadata
I20260924 21:12:43.378649 14434 ts_tablet_manager.cc:1939] T ae58a1db8198450fb2e8341f615697fb P 5ecda7f62fe647d78e4a990968bf5327: Deleting consensus metadata
I20260924 21:12:43.378769 13665 catalog_manager.cc:4841] TS d0982fc9b1f040af82515f52e5400c5f (127.6.210.130:34739): tablet ae58a1db8198450fb2e8341f615697fb (table test-table [id=1ca6a9ec81b246d3947be206ecb17cf3]) successfully deleted
I20260924 21:12:43.378957 13665 catalog_manager.cc:4841] TS 5ecda7f62fe647d78e4a990968bf5327 (127.6.210.131:37515): tablet ae58a1db8198450fb2e8341f615697fb (table test-table [id=1ca6a9ec81b246d3947be206ecb17cf3]) successfully deleted
W20260924 21:12:43.380916 14404 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:43.380730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:43821 (local address 127.0.0.1:35296)
0924 21:12:43.380780 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:43.380792 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:43.380842 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:43.380879 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.129:43821: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":34}
W20260924 21:12:43.381088 13666 catalog_manager.cc:4584] TS cb42303b63fc4ae2a46db49707af5edb (127.6.210.129:43821): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ae58a1db8198450fb2e8341f615697fb: Network error: Client connection negotiation failed: client connection to 127.6.210.129:43821: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260924 21:12:43.381183 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 14052
I20260924 21:12:43.386395 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 14188
I20260924 21:12:43.391762 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 13546
I20260924 21:12:43.396757 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 13734
2026-09-24T21:12:43Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4941 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:12:43 dist-test-slave-q276 krb5kdc[14439](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:12:43 dist-test-slave-q276 krb5kdc[14439](info): set up 2 sockets
Sep 24 21:12:43 dist-test-slave-q276 krb5kdc[14439](info): commencing operation
krb5kdc: starting...
W20260924 21:12:44.479009 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:12:44 dist-test-slave-q276 krb5kdc[14439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:12:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:12:44Z Disabled control of system clock
WARNING: no policy specified for oryx/127.6.210.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.210.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.210.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.190.keytab.
Entry for principal oryx/127.6.210.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.190.keytab.
I20260924 21:12:44.601619 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:34811
--webserver_interface=127.6.210.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.190.keytab
--principal=oryx/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:43967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:34811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:44.681036 14452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:44.681298 14452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:44.681339 14452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:44.682548 14452 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:12:44.682617 14452 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260924 21:12:44.682641 14452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:44.682663 14452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:44.682695 14452 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:12:44.682721 14452 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:12:44.684237 14452 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:43967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:34811
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.190.keytab
--principal=oryx/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:34811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:44.684514 14452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:44.684734 14452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 21:12:44.689693 14452 server_base.cc:1034] running on GCE node
W20260924 21:12:44.689646 14460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:44.689623 14458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:44.689617 14457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:44.690199 14452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:44.690554 14452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:44.691701 14452 hybrid_clock.cc:648] HybridClock initialized: now 1790284364691683 us; error 39 us; skew 500 ppm
Sep 24 21:12:44 dist-test-slave-q276 krb5kdc[14439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284364, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:44.694182 14452 init.cc:377] Logged in from keytab as oryx/127.6.210.190@KRBTEST.COM (short username oryx)
I20260924 21:12:44.694667 14452 webserver.cc:460] Webserver started at http://127.6.210.190:39677/ using document root <none> and password file <none>
I20260924 21:12:44.694849 14452 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:44.694887 14452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:44.694975 14452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:44.696002 14452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "e818eabd67334a1fab61630f7f4583e6"
format_stamp: "Formatted at 2026-09-24 21:12:44 on dist-test-slave-q276"
server_key: "ceaa6578db53a2dfdbe00428d21d5d4f"
server_key_iv: "5cd5b1e7f16d94b817cc508b9c9caa59"
server_key_version: "encryptionkey@0"
I20260924 21:12:44.696308 14452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "e818eabd67334a1fab61630f7f4583e6"
format_stamp: "Formatted at 2026-09-24 21:12:44 on dist-test-slave-q276"
server_key: "ceaa6578db53a2dfdbe00428d21d5d4f"
server_key_iv: "5cd5b1e7f16d94b817cc508b9c9caa59"
server_key_version: "encryptionkey@0"
I20260924 21:12:44.697528 14452 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:12:44.698165 14467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:44.698336 14452 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:12:44.698418 14452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "e818eabd67334a1fab61630f7f4583e6"
format_stamp: "Formatted at 2026-09-24 21:12:44 on dist-test-slave-q276"
server_key: "ceaa6578db53a2dfdbe00428d21d5d4f"
server_key_iv: "5cd5b1e7f16d94b817cc508b9c9caa59"
server_key_version: "encryptionkey@0"
I20260924 21:12:44.698470 14452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:44.709156 14452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:44.711001 14452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:44.711109 14452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:44.715521 14452 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:34811
I20260924 21:12:44.715626 14529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:34811 every 8 connection(s)
I20260924 21:12:44.715931 14452 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:12:44.716624 14530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:44.719063 14530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e818eabd67334a1fab61630f7f4583e6: Bootstrap starting.
I20260924 21:12:44.719657 14530 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e818eabd67334a1fab61630f7f4583e6: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:44.719956 14530 log.cc:824] T 00000000000000000000000000000000 P e818eabd67334a1fab61630f7f4583e6: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:44.720597 14530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e818eabd67334a1fab61630f7f4583e6: No bootstrap required, opened a new log
I20260924 21:12:44.722628 14530 raft_consensus.cc:348] T 00000000000000000000000000000000 P e818eabd67334a1fab61630f7f4583e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e818eabd67334a1fab61630f7f4583e6" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34811 } }
I20260924 21:12:44.722795 14530 raft_consensus.cc:374] T 00000000000000000000000000000000 P e818eabd67334a1fab61630f7f4583e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:44.722842 14530 raft_consensus.cc:729] T 00000000000000000000000000000000 P e818eabd67334a1fab61630f7f4583e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e818eabd67334a1fab61630f7f4583e6, State: Initialized, Role: FOLLOWER
I20260924 21:12:44.723002 14530 consensus_queue.cc:260] T 00000000000000000000000000000000 P e818eabd67334a1fab61630f7f4583e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e818eabd67334a1fab61630f7f4583e6" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34811 } }
I20260924 21:12:44.723085 14530 raft_consensus.cc:388] T 00000000000000000000000000000000 P e818eabd67334a1fab61630f7f4583e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:12:44.723127 14530 raft_consensus.cc:482] T 00000000000000000000000000000000 P e818eabd67334a1fab61630f7f4583e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:12:44.723176 14530 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e818eabd67334a1fab61630f7f4583e6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:44.723706 14530 raft_consensus.cc:504] T 00000000000000000000000000000000 P e818eabd67334a1fab61630f7f4583e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e818eabd67334a1fab61630f7f4583e6" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34811 } }
I20260924 21:12:44.723819 14530 leader_election.cc:302] T 00000000000000000000000000000000 P e818eabd67334a1fab61630f7f4583e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e818eabd67334a1fab61630f7f4583e6; no voters:
I20260924 21:12:44.723999 14530 leader_election.cc:288] T 00000000000000000000000000000000 P e818eabd67334a1fab61630f7f4583e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:12:44.724041 14533 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e818eabd67334a1fab61630f7f4583e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:44.724237 14533 raft_consensus.cc:686] T 00000000000000000000000000000000 P e818eabd67334a1fab61630f7f4583e6 [term 1 LEADER]: Becoming Leader. State: Replica: e818eabd67334a1fab61630f7f4583e6, State: Running, Role: LEADER
I20260924 21:12:44.724248 14530 sys_catalog.cc:564] T 00000000000000000000000000000000 P e818eabd67334a1fab61630f7f4583e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:12:44.724336 14533 consensus_queue.cc:237] T 00000000000000000000000000000000 P e818eabd67334a1fab61630f7f4583e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e818eabd67334a1fab61630f7f4583e6" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34811 } }
I20260924 21:12:44.724714 14534 sys_catalog.cc:455] T 00000000000000000000000000000000 P e818eabd67334a1fab61630f7f4583e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e818eabd67334a1fab61630f7f4583e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e818eabd67334a1fab61630f7f4583e6" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34811 } } }
I20260924 21:12:44.724784 14534 sys_catalog.cc:458] T 00000000000000000000000000000000 P e818eabd67334a1fab61630f7f4583e6 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:44.724694 14535 sys_catalog.cc:455] T 00000000000000000000000000000000 P e818eabd67334a1fab61630f7f4583e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e818eabd67334a1fab61630f7f4583e6. Latest consensus state: current_term: 1 leader_uuid: "e818eabd67334a1fab61630f7f4583e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e818eabd67334a1fab61630f7f4583e6" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34811 } } }
I20260924 21:12:44.724942 14535 sys_catalog.cc:458] T 00000000000000000000000000000000 P e818eabd67334a1fab61630f7f4583e6 [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:44.725265 14541 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:12:44.725693 14541 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:12:44.726152 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 14452
I20260924 21:12:44.726240 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:12:44.726459 6986 external_mini_cluster.cc:1428] Setting key e4804f52f17988f5f1ca2e02f8377765
I20260924 21:12:44.727334 14541 catalog_manager.cc:1347] Generated new cluster ID: 0a2710cda0ec40e6919e461ef8178d55
I20260924 21:12:44.727383 14541 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 21:12:44 dist-test-slave-q276 krb5kdc[14439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM
I20260924 21:12:44.742588 14541 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:12:44.743098 14541 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:12:44.745115 14553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:44.727030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35534 (local address 127.6.210.190:34811)
0924 21:12:44.727185 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:44.727196 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:44.730373 (+ 3177us) server_negotiation.cc:407] Connection header received
0924 21:12:44.730491 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:44.730498 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:44.730553 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:44.730682 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:44.731553 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:44.732961 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:44.734009 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:44.734678 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:44.734774 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:44.741270 (+ 6496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:44.741275 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:44.741276 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:44.741290 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:44.743470 (+ 2180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:44.743878 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:44.743881 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:44.743882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:44.743995 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:44.744196 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:44.744197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:44.744197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:44.744497 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:44.744638 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:44.744739 (+ 101us) server_negotiation.cc:299] Negotiation successful
0924 21:12:44.744892 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":84,"threads_started":1}
I20260924 21:12:44.751278 14541 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e818eabd67334a1fab61630f7f4583e6: Generated new TSK 0
I20260924 21:12:44.751451 14541 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.6.210.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.210.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.210.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.129.keytab.
Entry for principal oryx/127.6.210.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.129.keytab.
I20260924 21:12:44.781013 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.129:0
--local_ip_for_outbound_sockets=127.6.210.129
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:34811
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.129.keytab
--principal=oryx/127.6.210.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:43967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:44.861948 14556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:44.862169 14556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:44.862195 14556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:44.863374 14556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:44.863417 14556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:44.863437 14556 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 21:12:44.863495 14556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.129
I20260924 21:12:44.864956 14556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:43967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.129.keytab
--principal=oryx/127.6.210.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.210.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.210.190:34811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.129
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:44.865211 14556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:44.865444 14556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:44.870566 14562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:44.870611 14556 server_base.cc:1034] running on GCE node
W20260924 21:12:44.870620 14564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:44.870566 14561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:44.871047 14556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:44.871299 14556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:44.872440 14556 hybrid_clock.cc:648] HybridClock initialized: now 1790284364872428 us; error 28 us; skew 500 ppm
Sep 24 21:12:44 dist-test-slave-q276 krb5kdc[14439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284364, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:44.874969 14556 init.cc:377] Logged in from keytab as oryx/127.6.210.129@KRBTEST.COM (short username oryx)
I20260924 21:12:44.875447 14556 webserver.cc:460] Webserver started at http://127.6.210.129:43167/ using document root <none> and password file <none>
I20260924 21:12:44.875674 14556 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:44.875741 14556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:44.875892 14556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:44.876942 14556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-0/data/instance:
uuid: "6e7c05fe428048fdad95f46423ee18c1"
format_stamp: "Formatted at 2026-09-24 21:12:44 on dist-test-slave-q276"
server_key: "b965944ab1e079073ad246b88bbe751b"
server_key_iv: "f3828b5ef494014b070a49e6135f9951"
server_key_version: "encryptionkey@0"
I20260924 21:12:44.877316 14556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance:
uuid: "6e7c05fe428048fdad95f46423ee18c1"
format_stamp: "Formatted at 2026-09-24 21:12:44 on dist-test-slave-q276"
server_key: "b965944ab1e079073ad246b88bbe751b"
server_key_iv: "f3828b5ef494014b070a49e6135f9951"
server_key_version: "encryptionkey@0"
I20260924 21:12:44.878686 14556 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:12:44.879415 14571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:44.879572 14556 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:12:44.879670 14556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-0/wal
uuid: "6e7c05fe428048fdad95f46423ee18c1"
format_stamp: "Formatted at 2026-09-24 21:12:44 on dist-test-slave-q276"
server_key: "b965944ab1e079073ad246b88bbe751b"
server_key_iv: "f3828b5ef494014b070a49e6135f9951"
server_key_version: "encryptionkey@0"
I20260924 21:12:44.879748 14556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:44.925467 14556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:44.927675 14556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:44.927798 14556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:44.928265 14556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:44.928304 14556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:44.928357 14556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:44.928385 14556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:44.935346 14556 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.129:42319
I20260924 21:12:44.935417 14686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.129:42319 every 8 connection(s)
Sep 24 21:12:44 dist-test-slave-q276 krb5kdc[14439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284364, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM
I20260924 21:12:44.935781 14556 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
I20260924 21:12:44.936261 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 14556
I20260924 21:12:44.936373 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance
I20260924 21:12:44.936511 6986 external_mini_cluster.cc:1428] Setting key 934fbe609bca532d10f86c92a1945f31
I20260924 21:12:44.939744 14553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:44.928778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:33311 (local address 127.6.210.190:34811)
0924 21:12:44.928834 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:44.928836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:44.929090 (+ 254us) server_negotiation.cc:407] Connection header received
0924 21:12:44.929668 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:44.929670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:44.929683 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:44.929758 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:44.930687 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:44.931528 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:44.933172 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:44.933500 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:44.933585 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:44.936658 (+ 3073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:44.936660 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:44.936660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:44.936667 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:44.938391 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:44.938786 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:44.938787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:44.938788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:44.938855 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:44.939062 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:44.939063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:44.939063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:44.939218 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:44.939290 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:44.939543 (+ 253us) server_negotiation.cc:299] Negotiation successful
0924 21:12:44.939640 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 21:12:44.939924 14580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:44.928781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34811 (local address 127.6.210.129:33311)
0924 21:12:44.928962 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:44.928998 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:44.929516 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:44.929865 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:44.929870 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:44.930268 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:44.930581 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:44.930589 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:44.931623 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:44.931624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:44.933053 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:44.933057 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:44.933592 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:44.933593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:44.933691 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:44.934418 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:44.934429 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:44.936068 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:44.938484 (+ 2416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:44.938490 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:44.938491 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:44.938706 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:44.938920 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:44.938921 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:44.938921 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:44.939004 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:44.939290 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:44.939293 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:44.939403 (+ 110us) client_negotiation.cc:769] Sending connection context
0924 21:12:44.939516 (+ 113us) client_negotiation.cc:240] Negotiation successful
0924 21:12:44.939675 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":94,"threads_started":1}
I20260924 21:12:44.940438 14687 heartbeater.cc:347] Connected to a master server at 127.6.210.190:34811
I20260924 21:12:44.940521 14687 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:44.940714 14687 heartbeater.cc:511] Master 127.6.210.190:34811 requested a full tablet report, sending...
I20260924 21:12:44.941721 14483 ts_manager.cc:194] Registered new tserver with Master: 6e7c05fe428048fdad95f46423ee18c1 (127.6.210.129:42319)
I20260924 21:12:44.942747 14483 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.210.129@KRBTEST.COM'} at 127.6.210.129:33311
WARNING: no policy specified for oryx/127.6.210.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.210.130@KRBTEST.COM" created.
I20260924 21:12:44.947687 14694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:44.941483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34811 (local address 127.6.210.129:60851)
0924 21:12:44.941593 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:44.941603 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:44.941678 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:44.941865 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:44.941866 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:44.942027 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:44.942078 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:44.942079 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:44.942849 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:44.942850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:44.943660 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:44.943662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:44.944053 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:44.944054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:44.944119 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:44.944593 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:44.944597 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:44.944936 (+ 339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:44.946456 (+ 1520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:44.946457 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:44.946457 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:44.946682 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:44.946914 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:44.946915 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:44.946915 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:44.946967 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:44.947276 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:44.947277 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:44.947374 (+ 97us) client_negotiation.cc:769] Sending connection context
0924 21:12:44.947457 (+ 83us) client_negotiation.cc:240] Negotiation successful
0924 21:12:44.947570 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":64,"threads_started":1}
I20260924 21:12:44.947722 14553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:44.941470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:60851 (local address 127.6.210.190:34811)
0924 21:12:44.941506 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:44.941508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:44.941663 (+ 155us) server_negotiation.cc:407] Connection header received
0924 21:12:44.941744 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:44.941745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:44.941755 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:44.941805 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:44.942163 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:44.942719 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:44.943767 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:44.943969 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:44.944051 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:44.945016 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:44.945018 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:44.945018 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:44.945025 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:44.946386 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:44.946766 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:44.946767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:44.946768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:44.946853 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:44.947020 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:44.947021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:44.947021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:44.947197 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:44.947284 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:44.947514 (+ 230us) server_negotiation.cc:299] Negotiation successful
0924 21:12:44.947633 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.210.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.130.keytab.
Entry for principal oryx/127.6.210.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.130.keytab.
I20260924 21:12:44.964677 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.130:0
--local_ip_for_outbound_sockets=127.6.210.130
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:34811
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.130.keytab
--principal=oryx/127.6.210.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:43967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:45.045235 14696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:45.045439 14696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:45.045463 14696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:45.046643 14696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:45.046684 14696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:45.046705 14696 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 21:12:45.046754 14696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.130
I20260924 21:12:45.048254 14696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:43967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.130.keytab
--principal=oryx/127.6.210.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.210.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.210.190:34811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.130
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:45.048492 14696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:45.048734 14696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:45.053731 14704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:45.053731 14702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:45.053958 14696 server_base.cc:1034] running on GCE node
W20260924 21:12:45.053753 14701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:45.054237 14696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:45.054525 14696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:45.055681 14696 hybrid_clock.cc:648] HybridClock initialized: now 1790284365055670 us; error 27 us; skew 500 ppm
Sep 24 21:12:45 dist-test-slave-q276 krb5kdc[14439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284365, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:45.058339 14696 init.cc:377] Logged in from keytab as oryx/127.6.210.130@KRBTEST.COM (short username oryx)
I20260924 21:12:45.058763 14696 webserver.cc:460] Webserver started at http://127.6.210.130:37513/ using document root <none> and password file <none>
I20260924 21:12:45.058961 14696 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:45.059002 14696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:45.059092 14696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:45.060082 14696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-1/data/instance:
uuid: "5ce38422a138438180879808ba4a2850"
format_stamp: "Formatted at 2026-09-24 21:12:45 on dist-test-slave-q276"
server_key: "fff1e359ba438b5435d3ce062ab060d6"
server_key_iv: "0f47756fd92ad06a81e67f9f9794c429"
server_key_version: "encryptionkey@0"
I20260924 21:12:45.060397 14696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance:
uuid: "5ce38422a138438180879808ba4a2850"
format_stamp: "Formatted at 2026-09-24 21:12:45 on dist-test-slave-q276"
server_key: "fff1e359ba438b5435d3ce062ab060d6"
server_key_iv: "0f47756fd92ad06a81e67f9f9794c429"
server_key_version: "encryptionkey@0"
I20260924 21:12:45.061641 14696 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:12:45.062366 14711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:45.062541 14696 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:12:45.062613 14696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-1/wal
uuid: "5ce38422a138438180879808ba4a2850"
format_stamp: "Formatted at 2026-09-24 21:12:45 on dist-test-slave-q276"
server_key: "fff1e359ba438b5435d3ce062ab060d6"
server_key_iv: "0f47756fd92ad06a81e67f9f9794c429"
server_key_version: "encryptionkey@0"
I20260924 21:12:45.062665 14696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:45.073458 14696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:45.075342 14696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:45.075454 14696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:45.075981 14696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:45.076020 14696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:45.076042 14696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:45.076056 14696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 21:12:45 dist-test-slave-q276 krb5kdc[14439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284365, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM
I20260924 21:12:45.082938 14696 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.130:37161
I20260924 21:12:45.083123 14826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.130:37161 every 8 connection(s)
I20260924 21:12:45.083385 14696 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
I20260924 21:12:45.086064 14553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.076673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:41885 (local address 127.6.210.190:34811)
0924 21:12:45.076758 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:45.076760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:45.076976 (+ 216us) server_negotiation.cc:407] Connection header received
0924 21:12:45.077728 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:45.077730 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:45.077743 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:45.077826 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:45.078721 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.079594 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.080601 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.080950 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.081026 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.083449 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:45.083451 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:45.083451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:45.083460 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:45.084899 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:45.085299 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:45.085300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:45.085301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:45.085349 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:45.085562 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:45.085563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:45.085563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:45.085671 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:45.085719 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:45.085939 (+ 220us) server_negotiation.cc:299] Negotiation successful
0924 21:12:45.086001 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260924 21:12:45.086354 14720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.076650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34811 (local address 127.6.210.130:41885)
0924 21:12:45.076840 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:45.076880 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:45.077605 (+ 725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:45.077913 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:45.077918 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:45.078334 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:45.078617 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.078624 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.079702 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.079704 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.080529 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.080532 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.081051 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.081052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.081159 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.081718 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:45.081728 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:45.083339 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:45.084978 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:45.084983 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:45.084985 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:45.085229 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:45.085406 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:45.085407 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:45.085407 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:45.085509 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:45.085725 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:45.085729 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:45.085839 (+ 110us) client_negotiation.cc:769] Sending connection context
0924 21:12:45.085920 (+ 81us) client_negotiation.cc:240] Negotiation successful
0924 21:12:45.086036 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":120,"threads_started":1}
I20260924 21:12:45.086859 14827 heartbeater.cc:347] Connected to a master server at 127.6.210.190:34811
I20260924 21:12:45.086941 14827 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:45.087133 14827 heartbeater.cc:511] Master 127.6.210.190:34811 requested a full tablet report, sending...
I20260924 21:12:45.087538 14484 ts_manager.cc:194] Registered new tserver with Master: 5ce38422a138438180879808ba4a2850 (127.6.210.130:37161)
I20260924 21:12:45.088037 14484 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.210.130@KRBTEST.COM'} at 127.6.210.130:41885
I20260924 21:12:45.089327 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 14696
I20260924 21:12:45.089442 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance
I20260924 21:12:45.089641 6986 external_mini_cluster.cc:1428] Setting key d5dbc9739069a17e1ff9e42c009a4afc
I20260924 21:12:45.093812 14553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.088163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:57477 (local address 127.6.210.190:34811)
0924 21:12:45.088198 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:45.088200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:45.088208 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:12:45.088226 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:45.088226 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:45.088235 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:45.088292 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:45.088626 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.089132 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.090047 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.090329 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.090391 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.091377 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:45.091379 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:45.091379 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:45.091384 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:45.092752 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:45.093125 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:45.093126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:45.093126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:45.093173 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:45.093352 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:45.093353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:45.093353 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:45.093484 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:45.093534 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:45.093702 (+ 168us) server_negotiation.cc:299] Negotiation successful
0924 21:12:45.093755 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 21:12:45.093923 14833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.087846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34811 (local address 127.6.210.130:57477)
0924 21:12:45.087947 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:45.087956 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:45.087999 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:45.088314 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:45.088315 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:45.088491 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:45.088545 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.088546 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.089239 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.089240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.089979 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.089981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.090406 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.090407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.090480 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.090880 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:45.090885 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:45.091296 (+ 411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:45.092827 (+ 1531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:45.092828 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:45.092829 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:45.093055 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:45.093232 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:45.093233 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:45.093234 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:45.093298 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:45.093549 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:45.093550 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:45.093600 (+ 50us) client_negotiation.cc:769] Sending connection context
0924 21:12:45.093680 (+ 80us) client_negotiation.cc:240] Negotiation successful
0924 21:12:45.093795 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":62,"threads_started":1}
WARNING: no policy specified for oryx/127.6.210.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.210.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.210.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.131.keytab.
Entry for principal oryx/127.6.210.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.131.keytab.
I20260924 21:12:45.115483 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.131:0
--local_ip_for_outbound_sockets=127.6.210.131
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:34811
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.131.keytab
--principal=oryx/127.6.210.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:43967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:45.194680 14836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:45.194881 14836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:45.194905 14836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:45.196061 14836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:45.196105 14836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:45.196125 14836 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 21:12:45.196188 14836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.131
I20260924 21:12:45.197692 14836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:43967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.131.keytab
--principal=oryx/127.6.210.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.210.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.210.190:34811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.131
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:45.197922 14836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:45.198199 14836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:45.203102 14842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:45.203110 14844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:45.203117 14841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:45.203466 14836 server_base.cc:1034] running on GCE node
I20260924 21:12:45.203694 14836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:45.203930 14836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:45.205090 14836 hybrid_clock.cc:648] HybridClock initialized: now 1790284365205073 us; error 34 us; skew 500 ppm
Sep 24 21:12:45 dist-test-slave-q276 krb5kdc[14439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284365, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:45.207731 14836 init.cc:377] Logged in from keytab as oryx/127.6.210.131@KRBTEST.COM (short username oryx)
I20260924 21:12:45.208206 14836 webserver.cc:460] Webserver started at http://127.6.210.131:42633/ using document root <none> and password file <none>
I20260924 21:12:45.208446 14836 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:45.208513 14836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:45.208664 14836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:45.209729 14836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-2/data/instance:
uuid: "c4818c2f05694f8abb7074eee8f53398"
format_stamp: "Formatted at 2026-09-24 21:12:45 on dist-test-slave-q276"
server_key: "c50a825d7d672f3793db227eab7e8884"
server_key_iv: "de847af4d6a6806f3aaff91707307539"
server_key_version: "encryptionkey@0"
I20260924 21:12:45.210093 14836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance:
uuid: "c4818c2f05694f8abb7074eee8f53398"
format_stamp: "Formatted at 2026-09-24 21:12:45 on dist-test-slave-q276"
server_key: "c50a825d7d672f3793db227eab7e8884"
server_key_iv: "de847af4d6a6806f3aaff91707307539"
server_key_version: "encryptionkey@0"
I20260924 21:12:45.211508 14836 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:45.212291 14851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:45.212471 14836 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:12:45.212582 14836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-2/wal
uuid: "c4818c2f05694f8abb7074eee8f53398"
format_stamp: "Formatted at 2026-09-24 21:12:45 on dist-test-slave-q276"
server_key: "c50a825d7d672f3793db227eab7e8884"
server_key_iv: "de847af4d6a6806f3aaff91707307539"
server_key_version: "encryptionkey@0"
I20260924 21:12:45.212656 14836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:45.225955 14836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:45.227890 14836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:45.228049 14836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:45.228636 14836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:45.228686 14836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:45.228721 14836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:45.228739 14836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 21:12:45 dist-test-slave-q276 krb5kdc[14439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284365, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM
I20260924 21:12:45.235961 14836 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.131:35189
I20260924 21:12:45.236008 14966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.131:35189 every 8 connection(s)
I20260924 21:12:45.236402 14836 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
I20260924 21:12:45.238188 14553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.229034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:49177 (local address 127.6.210.190:34811)
0924 21:12:45.229087 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:45.229091 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:45.229341 (+ 250us) server_negotiation.cc:407] Connection header received
0924 21:12:45.229812 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:45.229814 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:45.229829 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:45.229918 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:45.230637 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.231140 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.232143 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.232348 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.232422 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.235309 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:45.235311 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:45.235311 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:45.235320 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:45.236998 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:45.237429 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:45.237429 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:45.237430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:45.237491 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:45.237669 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:45.237670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:45.237670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:45.237770 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:45.237819 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:45.238047 (+ 228us) server_negotiation.cc:299] Negotiation successful
0924 21:12:45.238106 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 21:12:45.238350 14860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.229029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34811 (local address 127.6.210.131:49177)
0924 21:12:45.229203 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:45.229243 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:45.229718 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:45.229998 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:45.230003 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:45.230347 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:45.230539 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.230544 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.231234 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.231235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.232029 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.232031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.232439 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.232440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.232555 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.233072 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:45.233081 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:45.235204 (+ 2123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:45.237069 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:45.237073 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:45.237075 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:45.237365 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:45.237534 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:45.237534 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:45.237535 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:45.237619 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:45.237825 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:45.237830 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:45.237941 (+ 111us) client_negotiation.cc:769] Sending connection context
0924 21:12:45.238021 (+ 80us) client_negotiation.cc:240] Negotiation successful
0924 21:12:45.238136 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":76,"threads_started":1}
I20260924 21:12:45.238986 14967 heartbeater.cc:347] Connected to a master server at 127.6.210.190:34811
I20260924 21:12:45.239058 14967 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:45.239241 14967 heartbeater.cc:511] Master 127.6.210.190:34811 requested a full tablet report, sending...
I20260924 21:12:45.239629 14484 ts_manager.cc:194] Registered new tserver with Master: c4818c2f05694f8abb7074eee8f53398 (127.6.210.131:35189)
I20260924 21:12:45.240180 14484 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.210.131@KRBTEST.COM'} at 127.6.210.131:49177
I20260924 21:12:45.240207 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 14836
I20260924 21:12:45.240341 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance
I20260924 21:12:45.240568 6986 external_mini_cluster.cc:1428] Setting key ef20a877574d051db9f108548154a2ae
I20260924 21:12:45.241384 6986 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:12:45.246328 14553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.240059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:39945 (local address 127.6.210.190:34811)
0924 21:12:45.240094 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:45.240095 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:45.240410 (+ 315us) server_negotiation.cc:407] Connection header received
0924 21:12:45.240516 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:45.240518 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:45.240548 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:45.240615 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:45.241010 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.241709 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.242479 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.242683 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.242765 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.243753 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:45.243754 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:45.243754 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:45.243760 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:45.245215 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:45.245561 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:45.245562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:45.245562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:45.245619 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:45.245816 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:45.245817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:45.245817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:45.245926 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:45.246000 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:45.246149 (+ 149us) server_negotiation.cc:299] Negotiation successful
0924 21:12:45.246222 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 21:12:45.246348 14973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.239995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34811 (local address 127.6.210.131:39945)
0924 21:12:45.240336 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:45.240348 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:45.240453 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:45.240625 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:45.240625 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:45.240848 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:45.240918 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.240920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.241805 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.241806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.242391 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.242393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.242766 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.242767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.242832 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.243255 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:45.243261 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:45.243668 (+ 407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:45.245306 (+ 1638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:45.245307 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:45.245308 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:45.245477 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:45.245685 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:45.245685 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:45.245686 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:45.245745 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:45.245992 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:45.245993 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:45.246037 (+ 44us) client_negotiation.cc:769] Sending connection context
0924 21:12:45.246124 (+ 87us) client_negotiation.cc:240] Negotiation successful
0924 21:12:45.246228 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":80,"threads_started":1}
Sep 24 21:12:45 dist-test-slave-q276 krb5kdc[14439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284365, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 21:12:45 dist-test-slave-q276 krb5kdc[14439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:45 dist-test-slave-q276 krb5kdc[14439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
W20260924 21:12:45.262974 14553 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:45.257577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35540 (local address 127.6.210.190:34811)
0924 21:12:45.257639 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:45.257642 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:45.257652 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:12:45.257675 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:45.257676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:45.257684 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:45.257732 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:45.258255 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.258772 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.259474 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.259650 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.259698 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.262872 (+ 3174us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35540: BlockingRecv error: recv got EOF from 127.0.0.1:35540 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 24 21:12:45 dist-test-slave-q276 krb5kdc[14439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284365, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM
I20260924 21:12:45.270078 14553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.263680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35548 (local address 127.6.210.190:34811)
0924 21:12:45.263755 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:45.263757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:45.263768 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:12:45.263846 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:45.263847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:45.263856 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:45.263911 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:45.264368 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.264863 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.265594 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.265790 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.265875 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.268030 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:45.268031 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:45.268031 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:45.268037 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:45.269198 (+ 1161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:45.269494 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:45.269494 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:45.269495 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:45.269542 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:45.269712 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:45.269712 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:45.269713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:45.269807 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:45.269879 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:45.269960 (+ 81us) server_negotiation.cc:299] Negotiation successful
0924 21:12:45.270016 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260924 21:12:45.271639 14484 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35548:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:12:45.273196 14484 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:12:45.279860 14989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.275527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:37161 (local address 127.0.0.1:57794)
0924 21:12:45.276020 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:45.276031 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:45.276181 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:45.276439 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:45.276440 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:45.276442 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:45.276507 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.276508 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.277389 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.277390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.278610 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.278612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.279551 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.279553 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.279629 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.279644 (+ 15us) client_negotiation.cc:769] Sending connection context
0924 21:12:45.279726 (+ 82us) client_negotiation.cc:240] Negotiation successful
0924 21:12:45.279780 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"mutex_wait_us":4,"thread_start_us":64,"threads_started":1}
I20260924 21:12:45.279896 14991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.275637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57794 (local address 127.6.210.130:37161)
0924 21:12:45.275838 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:45.275843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:45.276155 (+ 312us) server_negotiation.cc:407] Connection header received
0924 21:12:45.276270 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:45.276273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:45.276290 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:45.276345 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:45.276589 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.277309 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.278697 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.279467 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.279539 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.279618 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:45.279749 (+ 131us) server_negotiation.cc:299] Negotiation successful
0924 21:12:45.279792 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":90,"threads_started":1}
I20260924 21:12:45.280580 14990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.275527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:35189 (local address 127.0.0.1:34474)
0924 21:12:45.275666 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:45.275726 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:45.275796 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:45.276450 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:45.276451 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:45.276452 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:45.276499 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.276501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.277775 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.277776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.278968 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.278971 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.280248 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.280250 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.280311 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.280314 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:12:45.280380 (+ 66us) client_negotiation.cc:240] Negotiation successful
0924 21:12:45.280421 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":3,"thread_start_us":69,"threads_started":1}
I20260924 21:12:45.280886 14994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.275863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34474 (local address 127.6.210.131:35189)
0924 21:12:45.276030 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:45.276035 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:45.276045 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:12:45.276083 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:45.276085 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:45.276101 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:45.276221 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:45.276605 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.277662 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.279135 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.279893 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.280226 (+ 333us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.280312 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:45.280769 (+ 457us) server_negotiation.cc:299] Negotiation successful
0924 21:12:45.280820 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":78,"threads_started":1}
I20260924 21:12:45.280756 14761 tablet_service.cc:1467] Processing CreateTablet for tablet 002f23bbed8f410d83e548603b08e4dd (DEFAULT_TABLE table=test-table [id=ad81af3379db4bb59830c76658713cb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:45.281049 14992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.275732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:42319 (local address 127.0.0.1:40164)
0924 21:12:45.276024 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:45.276033 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:45.276095 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:45.276358 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:45.276361 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:45.276363 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:45.276425 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.276426 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.278311 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.278312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.279407 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.279410 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.280390 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.280392 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.280459 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.280463 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:12:45.280514 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 21:12:45.280552 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":65,"threads_started":1}
I20260924 21:12:45.281163 14761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 002f23bbed8f410d83e548603b08e4dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:45.281188 14993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.275847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40164 (local address 127.6.210.129:42319)
0924 21:12:45.276023 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:45.276028 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:45.276083 (+ 55us) server_negotiation.cc:407] Connection header received
0924 21:12:45.276249 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:45.276252 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:45.276267 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:45.276334 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:45.276504 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.277676 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.279520 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.280308 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.280384 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.280461 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:45.280628 (+ 167us) server_negotiation.cc:299] Negotiation successful
0924 21:12:45.280864 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":73,"threads_started":1}
I20260924 21:12:45.284456 14995 tablet_bootstrap.cc:492] T 002f23bbed8f410d83e548603b08e4dd P 5ce38422a138438180879808ba4a2850: Bootstrap starting.
I20260924 21:12:45.284977 14621 tablet_service.cc:1467] Processing CreateTablet for tablet 002f23bbed8f410d83e548603b08e4dd (DEFAULT_TABLE table=test-table [id=ad81af3379db4bb59830c76658713cb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:45.285126 14995 tablet_bootstrap.cc:654] T 002f23bbed8f410d83e548603b08e4dd P 5ce38422a138438180879808ba4a2850: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:45.285099 14901 tablet_service.cc:1467] Processing CreateTablet for tablet 002f23bbed8f410d83e548603b08e4dd (DEFAULT_TABLE table=test-table [id=ad81af3379db4bb59830c76658713cb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:45.285312 14621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 002f23bbed8f410d83e548603b08e4dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:45.285454 14901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 002f23bbed8f410d83e548603b08e4dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:45.285446 14995 log.cc:824] T 002f23bbed8f410d83e548603b08e4dd P 5ce38422a138438180879808ba4a2850: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:45.286197 14995 tablet_bootstrap.cc:492] T 002f23bbed8f410d83e548603b08e4dd P 5ce38422a138438180879808ba4a2850: No bootstrap required, opened a new log
I20260924 21:12:45.286324 14995 ts_tablet_manager.cc:1397] T 002f23bbed8f410d83e548603b08e4dd P 5ce38422a138438180879808ba4a2850: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:12:45.288638 14997 tablet_bootstrap.cc:492] T 002f23bbed8f410d83e548603b08e4dd P 6e7c05fe428048fdad95f46423ee18c1: Bootstrap starting.
I20260924 21:12:45.288627 14995 raft_consensus.cc:348] T 002f23bbed8f410d83e548603b08e4dd P 5ce38422a138438180879808ba4a2850 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } }
I20260924 21:12:45.288825 14995 raft_consensus.cc:374] T 002f23bbed8f410d83e548603b08e4dd P 5ce38422a138438180879808ba4a2850 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:45.288870 14995 raft_consensus.cc:729] T 002f23bbed8f410d83e548603b08e4dd P 5ce38422a138438180879808ba4a2850 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce38422a138438180879808ba4a2850, State: Initialized, Role: FOLLOWER
I20260924 21:12:45.289001 14995 consensus_queue.cc:260] T 002f23bbed8f410d83e548603b08e4dd P 5ce38422a138438180879808ba4a2850 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } }
I20260924 21:12:45.289062 14998 tablet_bootstrap.cc:492] T 002f23bbed8f410d83e548603b08e4dd P c4818c2f05694f8abb7074eee8f53398: Bootstrap starting.
I20260924 21:12:45.289270 14995 ts_tablet_manager.cc:1428] T 002f23bbed8f410d83e548603b08e4dd P 5ce38422a138438180879808ba4a2850: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:12:45.289355 14997 tablet_bootstrap.cc:654] T 002f23bbed8f410d83e548603b08e4dd P 6e7c05fe428048fdad95f46423ee18c1: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:45.289424 14827 heartbeater.cc:503] Master 127.6.210.190:34811 was elected leader, sending a full tablet report...
I20260924 21:12:45.289654 14998 tablet_bootstrap.cc:654] T 002f23bbed8f410d83e548603b08e4dd P c4818c2f05694f8abb7074eee8f53398: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:45.289680 14997 log.cc:824] T 002f23bbed8f410d83e548603b08e4dd P 6e7c05fe428048fdad95f46423ee18c1: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:45.289973 14998 log.cc:824] T 002f23bbed8f410d83e548603b08e4dd P c4818c2f05694f8abb7074eee8f53398: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:45.290746 14998 tablet_bootstrap.cc:492] T 002f23bbed8f410d83e548603b08e4dd P c4818c2f05694f8abb7074eee8f53398: No bootstrap required, opened a new log
I20260924 21:12:45.290803 14998 ts_tablet_manager.cc:1397] T 002f23bbed8f410d83e548603b08e4dd P c4818c2f05694f8abb7074eee8f53398: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:12:45.290769 14997 tablet_bootstrap.cc:492] T 002f23bbed8f410d83e548603b08e4dd P 6e7c05fe428048fdad95f46423ee18c1: No bootstrap required, opened a new log
I20260924 21:12:45.290853 14997 ts_tablet_manager.cc:1397] T 002f23bbed8f410d83e548603b08e4dd P 6e7c05fe428048fdad95f46423ee18c1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:12:45.292335 14998 raft_consensus.cc:348] T 002f23bbed8f410d83e548603b08e4dd P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } }
I20260924 21:12:45.292462 14998 raft_consensus.cc:374] T 002f23bbed8f410d83e548603b08e4dd P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:45.292485 14998 raft_consensus.cc:729] T 002f23bbed8f410d83e548603b08e4dd P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4818c2f05694f8abb7074eee8f53398, State: Initialized, Role: FOLLOWER
I20260924 21:12:45.292366 14997 raft_consensus.cc:348] T 002f23bbed8f410d83e548603b08e4dd P 6e7c05fe428048fdad95f46423ee18c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } }
I20260924 21:12:45.292511 14997 raft_consensus.cc:374] T 002f23bbed8f410d83e548603b08e4dd P 6e7c05fe428048fdad95f46423ee18c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:45.292560 14997 raft_consensus.cc:729] T 002f23bbed8f410d83e548603b08e4dd P 6e7c05fe428048fdad95f46423ee18c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e7c05fe428048fdad95f46423ee18c1, State: Initialized, Role: FOLLOWER
I20260924 21:12:45.292609 14998 consensus_queue.cc:260] T 002f23bbed8f410d83e548603b08e4dd P c4818c2f05694f8abb7074eee8f53398 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } }
I20260924 21:12:45.292682 14997 consensus_queue.cc:260] T 002f23bbed8f410d83e548603b08e4dd P 6e7c05fe428048fdad95f46423ee18c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } }
I20260924 21:12:45.292886 14998 ts_tablet_manager.cc:1428] T 002f23bbed8f410d83e548603b08e4dd P c4818c2f05694f8abb7074eee8f53398: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:12:45.292925 14997 ts_tablet_manager.cc:1428] T 002f23bbed8f410d83e548603b08e4dd P 6e7c05fe428048fdad95f46423ee18c1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:12:45.293004 14967 heartbeater.cc:503] Master 127.6.210.190:34811 was elected leader, sending a full tablet report...
I20260924 21:12:45.293272 14687 heartbeater.cc:503] Master 127.6.210.190:34811 was elected leader, sending a full tablet report...
W20260924 21:12:45.333917 14828 tablet.cc:2369] T 002f23bbed8f410d83e548603b08e4dd P 5ce38422a138438180879808ba4a2850: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:12:45.372483 15002 raft_consensus.cc:482] T 002f23bbed8f410d83e548603b08e4dd P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:12:45.372650 15002 raft_consensus.cc:504] T 002f23bbed8f410d83e548603b08e4dd P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } }
I20260924 21:12:45.373018 15002 leader_election.cc:288] T 002f23bbed8f410d83e548603b08e4dd P c4818c2f05694f8abb7074eee8f53398 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce38422a138438180879808ba4a2850 (127.6.210.130:37161), 6e7c05fe428048fdad95f46423ee18c1 (127.6.210.129:42319)
I20260924 21:12:45.376943 15004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.373359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:42319 (local address 127.6.210.131:44923)
0924 21:12:45.373544 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:45.373554 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:45.373607 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:45.373771 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:45.373771 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:45.373773 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:45.373828 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.373829 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.374677 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.374677 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.375902 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.375906 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.376638 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.376638 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.376742 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.376744 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 21:12:45.376804 (+ 60us) client_negotiation.cc:240] Negotiation successful
0924 21:12:45.376848 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":10,"thread_start_us":119,"threads_started":1}
I20260924 21:12:45.376974 14993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.373461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:44923 (local address 127.6.210.129:42319)
0924 21:12:45.373528 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:45.373530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:45.373603 (+ 73us) server_negotiation.cc:407] Connection header received
0924 21:12:45.373684 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:45.373685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:45.373695 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:45.373753 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:45.373897 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.374598 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.376000 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.376551 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.376632 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.376650 (+ 18us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:45.376849 (+ 199us) server_negotiation.cc:299] Negotiation successful
0924 21:12:45.376897 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 21:12:45.376947 14860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.373236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:37161 (local address 127.6.210.131:45465)
0924 21:12:45.373295 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:45.373308 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:45.373353 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:45.373551 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:45.373552 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:45.373554 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:45.373625 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.373627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.374598 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.374599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.375902 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.375906 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.376637 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.376638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.376740 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.376744 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:12:45.376804 (+ 60us) client_negotiation.cc:240] Negotiation successful
0924 21:12:45.376849 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":74}
I20260924 21:12:45.377030 14991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.373310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:45465 (local address 127.6.210.130:37161)
0924 21:12:45.373378 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:45.373380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:45.373420 (+ 40us) server_negotiation.cc:407] Connection header received
0924 21:12:45.373450 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:45.373450 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:45.373459 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:45.373527 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:45.373699 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.374519 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.376001 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.376561 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.376632 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.376640 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:45.376849 (+ 209us) server_negotiation.cc:299] Negotiation successful
0924 21:12:45.376903 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 21:12:45.377372 14781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "002f23bbed8f410d83e548603b08e4dd" candidate_uuid: "c4818c2f05694f8abb7074eee8f53398" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce38422a138438180879808ba4a2850" is_pre_election: true
I20260924 21:12:45.377379 14641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "002f23bbed8f410d83e548603b08e4dd" candidate_uuid: "c4818c2f05694f8abb7074eee8f53398" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e7c05fe428048fdad95f46423ee18c1" is_pre_election: true
I20260924 21:12:45.377561 14781 raft_consensus.cc:2454] T 002f23bbed8f410d83e548603b08e4dd P 5ce38422a138438180879808ba4a2850 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4818c2f05694f8abb7074eee8f53398 in term 0.
I20260924 21:12:45.377573 14641 raft_consensus.cc:2454] T 002f23bbed8f410d83e548603b08e4dd P 6e7c05fe428048fdad95f46423ee18c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4818c2f05694f8abb7074eee8f53398 in term 0.
I20260924 21:12:45.377832 14854 leader_election.cc:302] T 002f23bbed8f410d83e548603b08e4dd P c4818c2f05694f8abb7074eee8f53398 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e7c05fe428048fdad95f46423ee18c1, c4818c2f05694f8abb7074eee8f53398; no voters:
I20260924 21:12:45.377997 15002 raft_consensus.cc:2781] T 002f23bbed8f410d83e548603b08e4dd P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:12:45.378079 15002 raft_consensus.cc:482] T 002f23bbed8f410d83e548603b08e4dd P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:12:45.378130 15002 raft_consensus.cc:3037] T 002f23bbed8f410d83e548603b08e4dd P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:45.379141 15002 raft_consensus.cc:504] T 002f23bbed8f410d83e548603b08e4dd P c4818c2f05694f8abb7074eee8f53398 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } }
I20260924 21:12:45.379302 15002 leader_election.cc:288] T 002f23bbed8f410d83e548603b08e4dd P c4818c2f05694f8abb7074eee8f53398 [CANDIDATE]: Term 1 election: Requested vote from peers 5ce38422a138438180879808ba4a2850 (127.6.210.130:37161), 6e7c05fe428048fdad95f46423ee18c1 (127.6.210.129:42319)
I20260924 21:12:45.379458 14781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "002f23bbed8f410d83e548603b08e4dd" candidate_uuid: "c4818c2f05694f8abb7074eee8f53398" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce38422a138438180879808ba4a2850"
I20260924 21:12:45.379503 14641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "002f23bbed8f410d83e548603b08e4dd" candidate_uuid: "c4818c2f05694f8abb7074eee8f53398" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e7c05fe428048fdad95f46423ee18c1"
I20260924 21:12:45.379540 14781 raft_consensus.cc:3037] T 002f23bbed8f410d83e548603b08e4dd P 5ce38422a138438180879808ba4a2850 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:45.379606 14641 raft_consensus.cc:3037] T 002f23bbed8f410d83e548603b08e4dd P 6e7c05fe428048fdad95f46423ee18c1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:45.380280 14641 raft_consensus.cc:2454] T 002f23bbed8f410d83e548603b08e4dd P 6e7c05fe428048fdad95f46423ee18c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4818c2f05694f8abb7074eee8f53398 in term 1.
I20260924 21:12:45.380297 14781 raft_consensus.cc:2454] T 002f23bbed8f410d83e548603b08e4dd P 5ce38422a138438180879808ba4a2850 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4818c2f05694f8abb7074eee8f53398 in term 1.
I20260924 21:12:45.380445 14854 leader_election.cc:302] T 002f23bbed8f410d83e548603b08e4dd P c4818c2f05694f8abb7074eee8f53398 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce38422a138438180879808ba4a2850, c4818c2f05694f8abb7074eee8f53398; no voters:
I20260924 21:12:45.380566 15002 raft_consensus.cc:2781] T 002f23bbed8f410d83e548603b08e4dd P c4818c2f05694f8abb7074eee8f53398 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:45.380743 15002 raft_consensus.cc:686] T 002f23bbed8f410d83e548603b08e4dd P c4818c2f05694f8abb7074eee8f53398 [term 1 LEADER]: Becoming Leader. State: Replica: c4818c2f05694f8abb7074eee8f53398, State: Running, Role: LEADER
I20260924 21:12:45.380957 15002 consensus_queue.cc:237] T 002f23bbed8f410d83e548603b08e4dd P c4818c2f05694f8abb7074eee8f53398 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } }
I20260924 21:12:45.381844 14484 catalog_manager.cc:5495] T 002f23bbed8f410d83e548603b08e4dd P c4818c2f05694f8abb7074eee8f53398 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4818c2f05694f8abb7074eee8f53398 (127.6.210.131). New cstate: current_term: 1 leader_uuid: "c4818c2f05694f8abb7074eee8f53398" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } health_report { overall_health: UNKNOWN } } }
I20260924 21:12:45.395085 14994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.392813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34478 (local address 127.6.210.131:35189)
0924 21:12:45.392872 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:45.392874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:45.392883 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:12:45.392950 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:45.392950 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:45.392966 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:45.393029 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:12:45.393157 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.393610 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.394441 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.394619 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.394664 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.394822 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:12:45.394918 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:12:45.394975 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:45.395012 (+ 37us) server_negotiation.cc:299] Negotiation successful
0924 21:12:45.395029 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260924 21:12:45.396965 14781 raft_consensus.cc:1261] T 002f23bbed8f410d83e548603b08e4dd P 5ce38422a138438180879808ba4a2850 [term 1 FOLLOWER]: Refusing update from remote peer c4818c2f05694f8abb7074eee8f53398: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:45.396966 14641 raft_consensus.cc:1261] T 002f23bbed8f410d83e548603b08e4dd P 6e7c05fe428048fdad95f46423ee18c1 [term 1 FOLLOWER]: Refusing update from remote peer c4818c2f05694f8abb7074eee8f53398: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:45.397349 15005 consensus_queue.cc:1035] T 002f23bbed8f410d83e548603b08e4dd P c4818c2f05694f8abb7074eee8f53398 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:12:45.397485 15002 consensus_queue.cc:1035] T 002f23bbed8f410d83e548603b08e4dd P c4818c2f05694f8abb7074eee8f53398 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:12:45.400046 15010 mvcc.cc:205] Tried to move back new op lower bound from 7333004760663080960 to 7333004760601210880. Current Snapshot: MvccSnapshot[applied={T|T < 7333004760663080960}]
I20260924 21:12:45.402339 14483 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:12:45.402467 14483 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:12:45.403203 14483 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e818eabd67334a1fab61630f7f4583e6: Sending DeleteTablet for 3 replicas of tablet 002f23bbed8f410d83e548603b08e4dd
I20260924 21:12:45.403493 14901 tablet_service.cc:1514] Processing DeleteTablet for tablet 002f23bbed8f410d83e548603b08e4dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:45 UTC) from {username='oryx', principal='oryx/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:34474
I20260924 21:12:45.403578 14621 tablet_service.cc:1514] Processing DeleteTablet for tablet 002f23bbed8f410d83e548603b08e4dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:45 UTC) from {username='oryx', principal='oryx/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:40164
I20260924 21:12:45.403792 15019 tablet_replica.cc:330] T 002f23bbed8f410d83e548603b08e4dd P 6e7c05fe428048fdad95f46423ee18c1: stopping tablet replica
I20260924 21:12:45.403936 15019 raft_consensus.cc:2229] T 002f23bbed8f410d83e548603b08e4dd P 6e7c05fe428048fdad95f46423ee18c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:12:45.403915 14761 tablet_service.cc:1514] Processing DeleteTablet for tablet 002f23bbed8f410d83e548603b08e4dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:45 UTC) from {username='oryx', principal='oryx/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:57794
I20260924 21:12:45.404130 15019 raft_consensus.cc:2258] T 002f23bbed8f410d83e548603b08e4dd P 6e7c05fe428048fdad95f46423ee18c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:45.404132 15020 tablet_replica.cc:330] T 002f23bbed8f410d83e548603b08e4dd P 5ce38422a138438180879808ba4a2850: stopping tablet replica
I20260924 21:12:45.404227 15020 raft_consensus.cc:2229] T 002f23bbed8f410d83e548603b08e4dd P 5ce38422a138438180879808ba4a2850 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:12:45.404340 15020 raft_consensus.cc:2258] T 002f23bbed8f410d83e548603b08e4dd P 5ce38422a138438180879808ba4a2850 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:45.404338 15018 tablet_replica.cc:330] T 002f23bbed8f410d83e548603b08e4dd P c4818c2f05694f8abb7074eee8f53398: stopping tablet replica
I20260924 21:12:45.404299 14483 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35548:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 21:12:45.404418 15018 raft_consensus.cc:2229] T 002f23bbed8f410d83e548603b08e4dd P c4818c2f05694f8abb7074eee8f53398 [term 1 LEADER]: Raft consensus shutting down.
W20260924 21:12:45.404435 14483 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:12:45.404536 15018 raft_consensus.cc:2258] T 002f23bbed8f410d83e548603b08e4dd P c4818c2f05694f8abb7074eee8f53398 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:45.405000 15018 ts_tablet_manager.cc:1905] T 002f23bbed8f410d83e548603b08e4dd P c4818c2f05694f8abb7074eee8f53398: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:45.405458 15019 ts_tablet_manager.cc:1905] T 002f23bbed8f410d83e548603b08e4dd P 6e7c05fe428048fdad95f46423ee18c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:45.406422 14621 tablet_service.cc:1467] Processing CreateTablet for tablet 7d0daeaf270740bd88f41040660798d7 (DEFAULT_TABLE table=test-table [id=bca498d6da5046c391e9fc00d2f7695c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:45.406549 14621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d0daeaf270740bd88f41040660798d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:45.406759 15018 ts_tablet_manager.cc:1918] T 002f23bbed8f410d83e548603b08e4dd P c4818c2f05694f8abb7074eee8f53398: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:12:45.406842 15018 log.cc:1196] T 002f23bbed8f410d83e548603b08e4dd P c4818c2f05694f8abb7074eee8f53398: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-2/wal/wals/002f23bbed8f410d83e548603b08e4dd
I20260924 21:12:45.407115 15018 ts_tablet_manager.cc:1939] T 002f23bbed8f410d83e548603b08e4dd P c4818c2f05694f8abb7074eee8f53398: Deleting consensus metadata
I20260924 21:12:45.407503 14469 catalog_manager.cc:4841] TS c4818c2f05694f8abb7074eee8f53398 (127.6.210.131:35189): tablet 002f23bbed8f410d83e548603b08e4dd (table test-table [id=ad81af3379db4bb59830c76658713cb9]) successfully deleted
I20260924 21:12:45.407729 15019 ts_tablet_manager.cc:1918] T 002f23bbed8f410d83e548603b08e4dd P 6e7c05fe428048fdad95f46423ee18c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:12:45.407817 15019 log.cc:1196] T 002f23bbed8f410d83e548603b08e4dd P 6e7c05fe428048fdad95f46423ee18c1: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-0/wal/wals/002f23bbed8f410d83e548603b08e4dd
I20260924 21:12:45.407877 14761 tablet_service.cc:1467] Processing CreateTablet for tablet 7d0daeaf270740bd88f41040660798d7 (DEFAULT_TABLE table=test-table [id=bca498d6da5046c391e9fc00d2f7695c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:45.408000 14761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d0daeaf270740bd88f41040660798d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:45.408090 15019 ts_tablet_manager.cc:1939] T 002f23bbed8f410d83e548603b08e4dd P 6e7c05fe428048fdad95f46423ee18c1: Deleting consensus metadata
I20260924 21:12:45.409337 14995 tablet_bootstrap.cc:492] T 7d0daeaf270740bd88f41040660798d7 P 5ce38422a138438180879808ba4a2850: Bootstrap starting.
I20260924 21:12:45.409382 14901 tablet_service.cc:1467] Processing CreateTablet for tablet 7d0daeaf270740bd88f41040660798d7 (DEFAULT_TABLE table=test-table [id=bca498d6da5046c391e9fc00d2f7695c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:45.409485 14901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d0daeaf270740bd88f41040660798d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:45.409754 14995 tablet_bootstrap.cc:654] T 7d0daeaf270740bd88f41040660798d7 P 5ce38422a138438180879808ba4a2850: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:45.410356 14997 tablet_bootstrap.cc:492] T 7d0daeaf270740bd88f41040660798d7 P 6e7c05fe428048fdad95f46423ee18c1: Bootstrap starting.
I20260924 21:12:45.410382 14995 tablet_bootstrap.cc:492] T 7d0daeaf270740bd88f41040660798d7 P 5ce38422a138438180879808ba4a2850: No bootstrap required, opened a new log
I20260924 21:12:45.410418 14995 ts_tablet_manager.cc:1397] T 7d0daeaf270740bd88f41040660798d7 P 5ce38422a138438180879808ba4a2850: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:45.410570 14995 raft_consensus.cc:348] T 7d0daeaf270740bd88f41040660798d7 P 5ce38422a138438180879808ba4a2850 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.410636 14995 raft_consensus.cc:374] T 7d0daeaf270740bd88f41040660798d7 P 5ce38422a138438180879808ba4a2850 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:45.410657 14995 raft_consensus.cc:729] T 7d0daeaf270740bd88f41040660798d7 P 5ce38422a138438180879808ba4a2850 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce38422a138438180879808ba4a2850, State: Initialized, Role: FOLLOWER
I20260924 21:12:45.410709 14995 consensus_queue.cc:260] T 7d0daeaf270740bd88f41040660798d7 P 5ce38422a138438180879808ba4a2850 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.410759 14997 tablet_bootstrap.cc:654] T 7d0daeaf270740bd88f41040660798d7 P 6e7c05fe428048fdad95f46423ee18c1: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:45.410772 14998 tablet_bootstrap.cc:492] T 7d0daeaf270740bd88f41040660798d7 P c4818c2f05694f8abb7074eee8f53398: Bootstrap starting.
I20260924 21:12:45.410807 14995 ts_tablet_manager.cc:1428] T 7d0daeaf270740bd88f41040660798d7 P 5ce38422a138438180879808ba4a2850: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 21:12:45.411176 14998 tablet_bootstrap.cc:654] T 7d0daeaf270740bd88f41040660798d7 P c4818c2f05694f8abb7074eee8f53398: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:45.411391 14470 catalog_manager.cc:4841] TS 6e7c05fe428048fdad95f46423ee18c1 (127.6.210.129:42319): tablet 002f23bbed8f410d83e548603b08e4dd (table test-table [id=ad81af3379db4bb59830c76658713cb9]) successfully deleted
I20260924 21:12:45.411772 14997 tablet_bootstrap.cc:492] T 7d0daeaf270740bd88f41040660798d7 P 6e7c05fe428048fdad95f46423ee18c1: No bootstrap required, opened a new log
I20260924 21:12:45.411824 14997 ts_tablet_manager.cc:1397] T 7d0daeaf270740bd88f41040660798d7 P 6e7c05fe428048fdad95f46423ee18c1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:45.411891 15020 ts_tablet_manager.cc:1905] T 002f23bbed8f410d83e548603b08e4dd P 5ce38422a138438180879808ba4a2850: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:45.411965 14997 raft_consensus.cc:348] T 7d0daeaf270740bd88f41040660798d7 P 6e7c05fe428048fdad95f46423ee18c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.412040 14997 raft_consensus.cc:374] T 7d0daeaf270740bd88f41040660798d7 P 6e7c05fe428048fdad95f46423ee18c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:45.412061 14997 raft_consensus.cc:729] T 7d0daeaf270740bd88f41040660798d7 P 6e7c05fe428048fdad95f46423ee18c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e7c05fe428048fdad95f46423ee18c1, State: Initialized, Role: FOLLOWER
I20260924 21:12:45.412110 14997 consensus_queue.cc:260] T 7d0daeaf270740bd88f41040660798d7 P 6e7c05fe428048fdad95f46423ee18c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.412196 14997 ts_tablet_manager.cc:1428] T 7d0daeaf270740bd88f41040660798d7 P 6e7c05fe428048fdad95f46423ee18c1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 21:12:45.413430 15020 ts_tablet_manager.cc:1918] T 002f23bbed8f410d83e548603b08e4dd P 5ce38422a138438180879808ba4a2850: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:12:45.413501 15020 log.cc:1196] T 002f23bbed8f410d83e548603b08e4dd P 5ce38422a138438180879808ba4a2850: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-1/wal/wals/002f23bbed8f410d83e548603b08e4dd
I20260924 21:12:45.413735 15020 ts_tablet_manager.cc:1939] T 002f23bbed8f410d83e548603b08e4dd P 5ce38422a138438180879808ba4a2850: Deleting consensus metadata
I20260924 21:12:45.414147 14470 catalog_manager.cc:4841] TS 5ce38422a138438180879808ba4a2850 (127.6.210.130:37161): tablet 002f23bbed8f410d83e548603b08e4dd (table test-table [id=ad81af3379db4bb59830c76658713cb9]) successfully deleted
I20260924 21:12:45.414423 14998 tablet_bootstrap.cc:492] T 7d0daeaf270740bd88f41040660798d7 P c4818c2f05694f8abb7074eee8f53398: No bootstrap required, opened a new log
I20260924 21:12:45.414479 14998 ts_tablet_manager.cc:1397] T 7d0daeaf270740bd88f41040660798d7 P c4818c2f05694f8abb7074eee8f53398: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260924 21:12:45.414633 14998 raft_consensus.cc:348] T 7d0daeaf270740bd88f41040660798d7 P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.414711 14998 raft_consensus.cc:374] T 7d0daeaf270740bd88f41040660798d7 P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:45.414733 14998 raft_consensus.cc:729] T 7d0daeaf270740bd88f41040660798d7 P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4818c2f05694f8abb7074eee8f53398, State: Initialized, Role: FOLLOWER
I20260924 21:12:45.414781 14998 consensus_queue.cc:260] T 7d0daeaf270740bd88f41040660798d7 P c4818c2f05694f8abb7074eee8f53398 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.414865 14998 ts_tablet_manager.cc:1428] T 7d0daeaf270740bd88f41040660798d7 P c4818c2f05694f8abb7074eee8f53398: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
W20260924 21:12:45.436494 14688 tablet.cc:2369] T 7d0daeaf270740bd88f41040660798d7 P 6e7c05fe428048fdad95f46423ee18c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 21:12:45.486966 14968 tablet.cc:2369] T 7d0daeaf270740bd88f41040660798d7 P c4818c2f05694f8abb7074eee8f53398: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:12:45.557176 15013 raft_consensus.cc:482] T 7d0daeaf270740bd88f41040660798d7 P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:12:45.557329 15013 raft_consensus.cc:504] T 7d0daeaf270740bd88f41040660798d7 P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.557525 15013 leader_election.cc:288] T 7d0daeaf270740bd88f41040660798d7 P c4818c2f05694f8abb7074eee8f53398 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e7c05fe428048fdad95f46423ee18c1 (127.6.210.129:42319), 5ce38422a138438180879808ba4a2850 (127.6.210.130:37161)
I20260924 21:12:45.557789 14641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d0daeaf270740bd88f41040660798d7" candidate_uuid: "c4818c2f05694f8abb7074eee8f53398" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e7c05fe428048fdad95f46423ee18c1" is_pre_election: true
I20260924 21:12:45.557811 14781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d0daeaf270740bd88f41040660798d7" candidate_uuid: "c4818c2f05694f8abb7074eee8f53398" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce38422a138438180879808ba4a2850" is_pre_election: true
I20260924 21:12:45.557912 14641 raft_consensus.cc:2454] T 7d0daeaf270740bd88f41040660798d7 P 6e7c05fe428048fdad95f46423ee18c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4818c2f05694f8abb7074eee8f53398 in term 0.
I20260924 21:12:45.557911 14781 raft_consensus.cc:2454] T 7d0daeaf270740bd88f41040660798d7 P 5ce38422a138438180879808ba4a2850 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4818c2f05694f8abb7074eee8f53398 in term 0.
I20260924 21:12:45.558077 14854 leader_election.cc:302] T 7d0daeaf270740bd88f41040660798d7 P c4818c2f05694f8abb7074eee8f53398 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e7c05fe428048fdad95f46423ee18c1, c4818c2f05694f8abb7074eee8f53398; no voters:
I20260924 21:12:45.558173 15013 raft_consensus.cc:2781] T 7d0daeaf270740bd88f41040660798d7 P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:12:45.558216 15013 raft_consensus.cc:482] T 7d0daeaf270740bd88f41040660798d7 P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:12:45.558238 15013 raft_consensus.cc:3037] T 7d0daeaf270740bd88f41040660798d7 P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:45.558987 15013 raft_consensus.cc:504] T 7d0daeaf270740bd88f41040660798d7 P c4818c2f05694f8abb7074eee8f53398 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.559118 15013 leader_election.cc:288] T 7d0daeaf270740bd88f41040660798d7 P c4818c2f05694f8abb7074eee8f53398 [CANDIDATE]: Term 1 election: Requested vote from peers 6e7c05fe428048fdad95f46423ee18c1 (127.6.210.129:42319), 5ce38422a138438180879808ba4a2850 (127.6.210.130:37161)
I20260924 21:12:45.559280 14641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d0daeaf270740bd88f41040660798d7" candidate_uuid: "c4818c2f05694f8abb7074eee8f53398" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e7c05fe428048fdad95f46423ee18c1"
I20260924 21:12:45.559345 14781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d0daeaf270740bd88f41040660798d7" candidate_uuid: "c4818c2f05694f8abb7074eee8f53398" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce38422a138438180879808ba4a2850"
I20260924 21:12:45.559376 14641 raft_consensus.cc:3037] T 7d0daeaf270740bd88f41040660798d7 P 6e7c05fe428048fdad95f46423ee18c1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:45.559427 14781 raft_consensus.cc:3037] T 7d0daeaf270740bd88f41040660798d7 P 5ce38422a138438180879808ba4a2850 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:45.560030 14781 raft_consensus.cc:2454] T 7d0daeaf270740bd88f41040660798d7 P 5ce38422a138438180879808ba4a2850 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4818c2f05694f8abb7074eee8f53398 in term 1.
I20260924 21:12:45.560135 14641 raft_consensus.cc:2454] T 7d0daeaf270740bd88f41040660798d7 P 6e7c05fe428048fdad95f46423ee18c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4818c2f05694f8abb7074eee8f53398 in term 1.
I20260924 21:12:45.560225 14854 leader_election.cc:302] T 7d0daeaf270740bd88f41040660798d7 P c4818c2f05694f8abb7074eee8f53398 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce38422a138438180879808ba4a2850, c4818c2f05694f8abb7074eee8f53398; no voters:
I20260924 21:12:45.560320 15013 raft_consensus.cc:2781] T 7d0daeaf270740bd88f41040660798d7 P c4818c2f05694f8abb7074eee8f53398 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:45.560374 15013 raft_consensus.cc:686] T 7d0daeaf270740bd88f41040660798d7 P c4818c2f05694f8abb7074eee8f53398 [term 1 LEADER]: Becoming Leader. State: Replica: c4818c2f05694f8abb7074eee8f53398, State: Running, Role: LEADER
I20260924 21:12:45.560432 15013 consensus_queue.cc:237] T 7d0daeaf270740bd88f41040660798d7 P c4818c2f05694f8abb7074eee8f53398 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.561131 14483 catalog_manager.cc:5495] T 7d0daeaf270740bd88f41040660798d7 P c4818c2f05694f8abb7074eee8f53398 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4818c2f05694f8abb7074eee8f53398 (127.6.210.131). New cstate: current_term: 1 leader_uuid: "c4818c2f05694f8abb7074eee8f53398" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } health_report { overall_health: HEALTHY } } }
Sep 24 21:12:45 dist-test-slave-q276 krb5kdc[14439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284364, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.190@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM
I20260924 21:12:45.625957 14553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.619044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35564 (local address 127.6.210.190:34811)
0924 21:12:45.619145 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:45.619147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:45.619179 (+ 32us) server_negotiation.cc:407] Connection header received
0924 21:12:45.619256 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:45.619257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:45.619267 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:45.619325 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:45.619831 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.620374 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.621054 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.621231 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.621299 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.623420 (+ 2121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:45.623432 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:45.623433 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:45.623439 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:45.624700 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:45.625073 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:45.625074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:45.625075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:45.625151 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:45.625379 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:45.625379 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:45.625380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:45.625492 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:45.625566 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:45.625725 (+ 159us) server_negotiation.cc:299] Negotiation successful
0924 21:12:45.625844 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":69}
I20260924 21:12:45.626021 15030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.618968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34811 (local address 127.0.0.1:35564)
0924 21:12:45.619115 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:45.619125 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:45.619193 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:45.619344 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:45.619345 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:45.619662 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:45.619722 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.619724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.620446 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.620447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.620965 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.620966 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.621307 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.621308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.621345 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.621963 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:45.621968 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:45.623329 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:45.624799 (+ 1470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:45.624806 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:45.624809 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:45.624996 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:45.625215 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:45.625216 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:45.625217 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:45.625321 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:45.625557 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:45.625565 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:45.625609 (+ 44us) client_negotiation.cc:769] Sending connection context
0924 21:12:45.625701 (+ 92us) client_negotiation.cc:240] Negotiation successful
0924 21:12:45.625901 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":101,"thread_start_us":66,"threads_started":1}
I20260924 21:12:45.627988 14483 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:35564:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260924 21:12:45.628196 14483 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:12:45.629767 14621 tablet_service.cc:1467] Processing CreateTablet for tablet 9f6c2de37ca943f7860133079d1afece (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d8821d601cc243489ef0629acc012c1c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 21:12:45.629750 14761 tablet_service.cc:1467] Processing CreateTablet for tablet 9f6c2de37ca943f7860133079d1afece (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d8821d601cc243489ef0629acc012c1c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 21:12:45.629776 14901 tablet_service.cc:1467] Processing CreateTablet for tablet 9f6c2de37ca943f7860133079d1afece (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d8821d601cc243489ef0629acc012c1c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 21:12:45.629896 14621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f6c2de37ca943f7860133079d1afece. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:45.629904 14901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f6c2de37ca943f7860133079d1afece. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:45.629904 14761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f6c2de37ca943f7860133079d1afece. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:45.631379 14995 tablet_bootstrap.cc:492] T 9f6c2de37ca943f7860133079d1afece P 5ce38422a138438180879808ba4a2850: Bootstrap starting.
I20260924 21:12:45.631505 14997 tablet_bootstrap.cc:492] T 9f6c2de37ca943f7860133079d1afece P 6e7c05fe428048fdad95f46423ee18c1: Bootstrap starting.
I20260924 21:12:45.631795 14995 tablet_bootstrap.cc:654] T 9f6c2de37ca943f7860133079d1afece P 5ce38422a138438180879808ba4a2850: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:45.631884 14997 tablet_bootstrap.cc:654] T 9f6c2de37ca943f7860133079d1afece P 6e7c05fe428048fdad95f46423ee18c1: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:45.632161 14998 tablet_bootstrap.cc:492] T 9f6c2de37ca943f7860133079d1afece P c4818c2f05694f8abb7074eee8f53398: Bootstrap starting.
I20260924 21:12:45.632491 14998 tablet_bootstrap.cc:654] T 9f6c2de37ca943f7860133079d1afece P c4818c2f05694f8abb7074eee8f53398: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:45.632499 14997 tablet_bootstrap.cc:492] T 9f6c2de37ca943f7860133079d1afece P 6e7c05fe428048fdad95f46423ee18c1: No bootstrap required, opened a new log
I20260924 21:12:45.632558 14997 ts_tablet_manager.cc:1397] T 9f6c2de37ca943f7860133079d1afece P 6e7c05fe428048fdad95f46423ee18c1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:45.632704 14997 raft_consensus.cc:348] T 9f6c2de37ca943f7860133079d1afece P 6e7c05fe428048fdad95f46423ee18c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.632822 14997 raft_consensus.cc:374] T 9f6c2de37ca943f7860133079d1afece P 6e7c05fe428048fdad95f46423ee18c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:45.632845 14997 raft_consensus.cc:729] T 9f6c2de37ca943f7860133079d1afece P 6e7c05fe428048fdad95f46423ee18c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e7c05fe428048fdad95f46423ee18c1, State: Initialized, Role: FOLLOWER
I20260924 21:12:45.632879 14995 tablet_bootstrap.cc:492] T 9f6c2de37ca943f7860133079d1afece P 5ce38422a138438180879808ba4a2850: No bootstrap required, opened a new log
I20260924 21:12:45.632896 14997 consensus_queue.cc:260] T 9f6c2de37ca943f7860133079d1afece P 6e7c05fe428048fdad95f46423ee18c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.632925 14995 ts_tablet_manager.cc:1397] T 9f6c2de37ca943f7860133079d1afece P 5ce38422a138438180879808ba4a2850: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:12:45.633013 14997 ts_tablet_manager.cc:1428] T 9f6c2de37ca943f7860133079d1afece P 6e7c05fe428048fdad95f46423ee18c1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 21:12:45.633061 14995 raft_consensus.cc:348] T 9f6c2de37ca943f7860133079d1afece P 5ce38422a138438180879808ba4a2850 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.633046 15003 tablet_replica.cc:439] T 9f6c2de37ca943f7860133079d1afece P 6e7c05fe428048fdad95f46423ee18c1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } } }
I20260924 21:12:45.633131 14995 raft_consensus.cc:374] T 9f6c2de37ca943f7860133079d1afece P 5ce38422a138438180879808ba4a2850 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:45.633152 14995 raft_consensus.cc:729] T 9f6c2de37ca943f7860133079d1afece P 5ce38422a138438180879808ba4a2850 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce38422a138438180879808ba4a2850, State: Initialized, Role: FOLLOWER
I20260924 21:12:45.633142 15003 tablet_replica.cc:442] T 9f6c2de37ca943f7860133079d1afece P 6e7c05fe428048fdad95f46423ee18c1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:12:45.633195 14995 consensus_queue.cc:260] T 9f6c2de37ca943f7860133079d1afece P 5ce38422a138438180879808ba4a2850 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.633294 14995 ts_tablet_manager.cc:1428] T 9f6c2de37ca943f7860133079d1afece P 5ce38422a138438180879808ba4a2850: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:45.633443 14999 tablet_replica.cc:439] T 9f6c2de37ca943f7860133079d1afece P 5ce38422a138438180879808ba4a2850: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } } }
I20260924 21:12:45.633523 14999 tablet_replica.cc:442] T 9f6c2de37ca943f7860133079d1afece P 5ce38422a138438180879808ba4a2850: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:12:45.633790 14998 tablet_bootstrap.cc:492] T 9f6c2de37ca943f7860133079d1afece P c4818c2f05694f8abb7074eee8f53398: No bootstrap required, opened a new log
I20260924 21:12:45.633837 14998 ts_tablet_manager.cc:1397] T 9f6c2de37ca943f7860133079d1afece P c4818c2f05694f8abb7074eee8f53398: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:12:45.633947 14998 raft_consensus.cc:348] T 9f6c2de37ca943f7860133079d1afece P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.634017 14998 raft_consensus.cc:374] T 9f6c2de37ca943f7860133079d1afece P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:45.634035 14998 raft_consensus.cc:729] T 9f6c2de37ca943f7860133079d1afece P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4818c2f05694f8abb7074eee8f53398, State: Initialized, Role: FOLLOWER
I20260924 21:12:45.634099 14998 consensus_queue.cc:260] T 9f6c2de37ca943f7860133079d1afece P c4818c2f05694f8abb7074eee8f53398 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.634198 14998 ts_tablet_manager.cc:1428] T 9f6c2de37ca943f7860133079d1afece P c4818c2f05694f8abb7074eee8f53398: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:45.634219 15013 tablet_replica.cc:439] T 9f6c2de37ca943f7860133079d1afece P c4818c2f05694f8abb7074eee8f53398: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } } }
I20260924 21:12:45.634343 15013 tablet_replica.cc:442] T 9f6c2de37ca943f7860133079d1afece P c4818c2f05694f8abb7074eee8f53398: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:12:45.647624 14999 raft_consensus.cc:482] T 9f6c2de37ca943f7860133079d1afece P 5ce38422a138438180879808ba4a2850 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:12:45.647722 14999 raft_consensus.cc:504] T 9f6c2de37ca943f7860133079d1afece P 5ce38422a138438180879808ba4a2850 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.648063 14999 leader_election.cc:288] T 9f6c2de37ca943f7860133079d1afece P 5ce38422a138438180879808ba4a2850 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e7c05fe428048fdad95f46423ee18c1 (127.6.210.129:42319), c4818c2f05694f8abb7074eee8f53398 (127.6.210.131:35189)
I20260924 21:12:45.649685 15003 raft_consensus.cc:482] T 9f6c2de37ca943f7860133079d1afece P 6e7c05fe428048fdad95f46423ee18c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:12:45.649768 15003 raft_consensus.cc:504] T 9f6c2de37ca943f7860133079d1afece P 6e7c05fe428048fdad95f46423ee18c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.650082 15003 leader_election.cc:288] T 9f6c2de37ca943f7860133079d1afece P 6e7c05fe428048fdad95f46423ee18c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce38422a138438180879808ba4a2850 (127.6.210.130:37161), c4818c2f05694f8abb7074eee8f53398 (127.6.210.131:35189)
I20260924 21:12:45.651707 14993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.648502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:59359 (local address 127.6.210.129:42319)
0924 21:12:45.648550 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:45.648552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:45.648567 (+ 15us) server_negotiation.cc:407] Connection header received
0924 21:12:45.648605 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:45.648606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:45.648615 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:45.648668 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:45.648855 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.649347 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.650774 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.651312 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.651356 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.651365 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:45.651583 (+ 218us) server_negotiation.cc:299] Negotiation successful
0924 21:12:45.651631 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 21:12:45.651738 15034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.648249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:35189 (local address 127.6.210.130:45319)
0924 21:12:45.648374 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:45.648404 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:45.648464 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:45.648631 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:45.648632 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:45.648634 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:45.648714 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.648716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.649355 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.649356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.650798 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.650800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.651512 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.651513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.651578 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.651580 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 21:12:45.651624 (+ 44us) client_negotiation.cc:240] Negotiation successful
0924 21:12:45.651660 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
I20260924 21:12:45.651997 14994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.648437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:45319 (local address 127.6.210.131:35189)
0924 21:12:45.648507 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:45.648509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:45.648518 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:12:45.648543 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:45.648544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:45.648557 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:45.648618 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:45.648785 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.649275 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.650871 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.651419 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.651477 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.651485 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:45.651883 (+ 398us) server_negotiation.cc:299] Negotiation successful
0924 21:12:45.651926 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260924 21:12:45.652176 14917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f6c2de37ca943f7860133079d1afece" candidate_uuid: "5ce38422a138438180879808ba4a2850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4818c2f05694f8abb7074eee8f53398" is_pre_election: true
I20260924 21:12:45.652318 14917 raft_consensus.cc:2454] T 9f6c2de37ca943f7860133079d1afece P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce38422a138438180879808ba4a2850 in term 0.
I20260924 21:12:45.652400 15035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.648251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:42319 (local address 127.6.210.130:59359)
0924 21:12:45.648438 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:45.648473 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:45.648529 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:45.648691 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:45.648692 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:45.648694 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:45.648781 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.648783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.649434 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.649435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.650687 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.650689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.651424 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.651425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.651502 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.651506 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:12:45.651825 (+ 319us) client_negotiation.cc:240] Negotiation successful
0924 21:12:45.652035 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":3,"thread_start_us":72,"threads_started":1}
I20260924 21:12:45.652593 14713 leader_election.cc:302] T 9f6c2de37ca943f7860133079d1afece P 5ce38422a138438180879808ba4a2850 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce38422a138438180879808ba4a2850, c4818c2f05694f8abb7074eee8f53398; no voters:
I20260924 21:12:45.652601 14641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f6c2de37ca943f7860133079d1afece" candidate_uuid: "5ce38422a138438180879808ba4a2850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e7c05fe428048fdad95f46423ee18c1" is_pre_election: true
I20260924 21:12:45.652678 14641 raft_consensus.cc:2454] T 9f6c2de37ca943f7860133079d1afece P 6e7c05fe428048fdad95f46423ee18c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce38422a138438180879808ba4a2850 in term 0.
I20260924 21:12:45.652709 14999 raft_consensus.cc:2781] T 9f6c2de37ca943f7860133079d1afece P 5ce38422a138438180879808ba4a2850 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:12:45.652765 14999 raft_consensus.cc:482] T 9f6c2de37ca943f7860133079d1afece P 5ce38422a138438180879808ba4a2850 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:12:45.652786 14999 raft_consensus.cc:3037] T 9f6c2de37ca943f7860133079d1afece P 5ce38422a138438180879808ba4a2850 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:45.653656 14999 raft_consensus.cc:504] T 9f6c2de37ca943f7860133079d1afece P 5ce38422a138438180879808ba4a2850 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.653784 14999 leader_election.cc:288] T 9f6c2de37ca943f7860133079d1afece P 5ce38422a138438180879808ba4a2850 [CANDIDATE]: Term 1 election: Requested vote from peers 6e7c05fe428048fdad95f46423ee18c1 (127.6.210.129:42319), c4818c2f05694f8abb7074eee8f53398 (127.6.210.131:35189)
I20260924 21:12:45.653952 14641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f6c2de37ca943f7860133079d1afece" candidate_uuid: "5ce38422a138438180879808ba4a2850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e7c05fe428048fdad95f46423ee18c1"
I20260924 21:12:45.653985 14917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f6c2de37ca943f7860133079d1afece" candidate_uuid: "5ce38422a138438180879808ba4a2850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4818c2f05694f8abb7074eee8f53398"
I20260924 21:12:45.654022 14641 raft_consensus.cc:3037] T 9f6c2de37ca943f7860133079d1afece P 6e7c05fe428048fdad95f46423ee18c1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:45.654047 14917 raft_consensus.cc:3037] T 9f6c2de37ca943f7860133079d1afece P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:45.654259 15038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.650488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:35949 (local address 127.6.210.131:35189)
0924 21:12:45.650918 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:45.650920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:45.651022 (+ 102us) server_negotiation.cc:407] Connection header received
0924 21:12:45.651110 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:45.651111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:45.651120 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:45.651158 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:45.651341 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.652196 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.653445 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.653878 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.653951 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.653960 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:45.654155 (+ 195us) server_negotiation.cc:299] Negotiation successful
0924 21:12:45.654200 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":92,"threads_started":1}
I20260924 21:12:45.654469 15037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.650418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:35189 (local address 127.6.210.129:35949)
0924 21:12:45.650971 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:45.650981 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:45.651043 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:45.651212 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:45.651213 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:45.651214 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:45.651273 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.651275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.652286 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.652287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.653352 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.653355 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.653990 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.653991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.654071 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.654074 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:12:45.654337 (+ 263us) client_negotiation.cc:240] Negotiation successful
0924 21:12:45.654379 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":57,"threads_started":1}
I20260924 21:12:45.654759 14921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f6c2de37ca943f7860133079d1afece" candidate_uuid: "6e7c05fe428048fdad95f46423ee18c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4818c2f05694f8abb7074eee8f53398" is_pre_election: true
I20260924 21:12:45.654888 14917 raft_consensus.cc:2454] T 9f6c2de37ca943f7860133079d1afece P c4818c2f05694f8abb7074eee8f53398 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce38422a138438180879808ba4a2850 in term 1.
I20260924 21:12:45.654938 14641 raft_consensus.cc:2454] T 9f6c2de37ca943f7860133079d1afece P 6e7c05fe428048fdad95f46423ee18c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce38422a138438180879808ba4a2850 in term 1.
I20260924 21:12:45.655033 14713 leader_election.cc:302] T 9f6c2de37ca943f7860133079d1afece P 5ce38422a138438180879808ba4a2850 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce38422a138438180879808ba4a2850, c4818c2f05694f8abb7074eee8f53398; no voters:
I20260924 21:12:45.655133 14999 raft_consensus.cc:2781] T 9f6c2de37ca943f7860133079d1afece P 5ce38422a138438180879808ba4a2850 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:45.655277 14999 raft_consensus.cc:686] T 9f6c2de37ca943f7860133079d1afece P 5ce38422a138438180879808ba4a2850 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce38422a138438180879808ba4a2850, State: Running, Role: LEADER
I20260924 21:12:45.655344 14999 consensus_queue.cc:237] T 9f6c2de37ca943f7860133079d1afece P 5ce38422a138438180879808ba4a2850 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.655418 14991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.650290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:41817 (local address 127.6.210.130:37161)
0924 21:12:45.650324 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:45.650325 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:45.650635 (+ 310us) server_negotiation.cc:407] Connection header received
0924 21:12:45.650688 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:45.650688 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:45.650698 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:45.650753 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:12:45.650900 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.651588 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.654178 (+ 2590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.654651 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.654698 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.654714 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:45.655320 (+ 606us) server_negotiation.cc:299] Negotiation successful
0924 21:12:45.655360 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 21:12:45.655444 15036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.650232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:37161 (local address 127.6.210.129:41817)
0924 21:12:45.650572 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:45.650583 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:45.650641 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:45.650759 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:45.650759 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:45.650761 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:12:45.650816 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.650818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.651673 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.651674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.654084 (+ 2410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.654087 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.655155 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.655156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.655222 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.655225 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:12:45.655307 (+ 82us) client_negotiation.cc:240] Negotiation successful
0924 21:12:45.655367 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":73,"thread_start_us":72,"threads_started":1}
I20260924 21:12:45.655663 14781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f6c2de37ca943f7860133079d1afece" candidate_uuid: "6e7c05fe428048fdad95f46423ee18c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce38422a138438180879808ba4a2850" is_pre_election: true
I20260924 21:12:45.655829 15039 tablet_replica.cc:439] T 9f6c2de37ca943f7860133079d1afece P 5ce38422a138438180879808ba4a2850: TxnStatusTablet state changed. Reason: New leader 5ce38422a138438180879808ba4a2850. Latest consensus state: current_term: 1 leader_uuid: "5ce38422a138438180879808ba4a2850" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } } }
I20260924 21:12:45.655912 15039 tablet_replica.cc:442] T 9f6c2de37ca943f7860133079d1afece P 5ce38422a138438180879808ba4a2850: This TxnStatusTablet replica's current role is: LEADER
I20260924 21:12:45.655876 14574 leader_election.cc:302] T 9f6c2de37ca943f7860133079d1afece P 6e7c05fe428048fdad95f46423ee18c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6e7c05fe428048fdad95f46423ee18c1; no voters: 5ce38422a138438180879808ba4a2850, c4818c2f05694f8abb7074eee8f53398
I20260924 21:12:45.656037 15003 raft_consensus.cc:2726] T 9f6c2de37ca943f7860133079d1afece P 6e7c05fe428048fdad95f46423ee18c1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 21:12:45.656025 14484 catalog_manager.cc:5495] T 9f6c2de37ca943f7860133079d1afece P 5ce38422a138438180879808ba4a2850 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ce38422a138438180879808ba4a2850 (127.6.210.130). New cstate: current_term: 1 leader_uuid: "5ce38422a138438180879808ba4a2850" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } health_report { overall_health: UNKNOWN } } }
I20260924 21:12:45.656185 15041 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260924 21:12:45.656296 15041 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260924 21:12:45.662995 15030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.660253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:37161 (local address 127.0.0.1:57806)
0924 21:12:45.660334 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:45.660343 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:45.660385 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:45.660535 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:45.660536 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:45.660537 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 21:12:45.660575 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.660577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.661407 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.661408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.661968 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.661969 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.662460 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.662461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.662533 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.662546 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 21:12:45.662833 (+ 287us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 21:12:45.662841 (+ 8us) client_negotiation.cc:769] Sending connection context
0924 21:12:45.662896 (+ 55us) client_negotiation.cc:240] Negotiation successful
0924 21:12:45.662937 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260924 21:12:45.662997 14991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.660299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57806 (local address 127.6.210.130:37161)
0924 21:12:45.660339 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:45.660341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:45.660374 (+ 33us) server_negotiation.cc:407] Connection header received
0924 21:12:45.660439 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:45.660440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:45.660457 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:45.660537 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:12:45.660612 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.661323 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.662051 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.662373 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.662438 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.662637 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:12:45.662759 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:12:45.662840 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:45.662899 (+ 59us) server_negotiation.cc:299] Negotiation successful
0924 21:12:45.662920 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 21:12:45.664175 14641 raft_consensus.cc:1261] T 9f6c2de37ca943f7860133079d1afece P 6e7c05fe428048fdad95f46423ee18c1 [term 1 FOLLOWER]: Refusing update from remote peer 5ce38422a138438180879808ba4a2850: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:45.664258 14921 raft_consensus.cc:1261] T 9f6c2de37ca943f7860133079d1afece P c4818c2f05694f8abb7074eee8f53398 [term 1 FOLLOWER]: Refusing update from remote peer 5ce38422a138438180879808ba4a2850: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:45.664436 14999 consensus_queue.cc:1035] T 9f6c2de37ca943f7860133079d1afece P 5ce38422a138438180879808ba4a2850 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:12:45.664659 14999 consensus_queue.cc:1035] T 9f6c2de37ca943f7860133079d1afece P 5ce38422a138438180879808ba4a2850 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:12:45.664978 15003 tablet_replica.cc:439] T 9f6c2de37ca943f7860133079d1afece P 6e7c05fe428048fdad95f46423ee18c1: TxnStatusTablet state changed. Reason: New leader 5ce38422a138438180879808ba4a2850. Latest consensus state: current_term: 1 leader_uuid: "5ce38422a138438180879808ba4a2850" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } } }
I20260924 21:12:45.665079 15003 tablet_replica.cc:442] T 9f6c2de37ca943f7860133079d1afece P 6e7c05fe428048fdad95f46423ee18c1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:12:45.665264 15013 tablet_replica.cc:439] T 9f6c2de37ca943f7860133079d1afece P c4818c2f05694f8abb7074eee8f53398: TxnStatusTablet state changed. Reason: New leader 5ce38422a138438180879808ba4a2850. Latest consensus state: current_term: 1 leader_uuid: "5ce38422a138438180879808ba4a2850" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } } }
I20260924 21:12:45.665359 15013 tablet_replica.cc:442] T 9f6c2de37ca943f7860133079d1afece P c4818c2f05694f8abb7074eee8f53398: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:12:45.666034 15039 tablet_replica.cc:439] T 9f6c2de37ca943f7860133079d1afece P 5ce38422a138438180879808ba4a2850: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5ce38422a138438180879808ba4a2850" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } } }
I20260924 21:12:45.666123 15039 tablet_replica.cc:442] T 9f6c2de37ca943f7860133079d1afece P 5ce38422a138438180879808ba4a2850: This TxnStatusTablet replica's current role is: LEADER
I20260924 21:12:45.666426 15003 tablet_replica.cc:439] T 9f6c2de37ca943f7860133079d1afece P 6e7c05fe428048fdad95f46423ee18c1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5ce38422a138438180879808ba4a2850" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } } }
I20260924 21:12:45.666512 15003 tablet_replica.cc:442] T 9f6c2de37ca943f7860133079d1afece P 6e7c05fe428048fdad95f46423ee18c1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:12:45.667605 14999 tablet_replica.cc:439] T 9f6c2de37ca943f7860133079d1afece P 5ce38422a138438180879808ba4a2850: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5ce38422a138438180879808ba4a2850" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } } }
I20260924 21:12:45.667694 14999 tablet_replica.cc:442] T 9f6c2de37ca943f7860133079d1afece P 5ce38422a138438180879808ba4a2850: This TxnStatusTablet replica's current role is: LEADER
I20260924 21:12:45.667814 15008 mvcc.cc:205] Tried to move back new op lower bound from 7333004761758523392 to 7333004761724878848. Current Snapshot: MvccSnapshot[applied={T|T < 7333004761758523392}]
I20260924 21:12:45.667896 15013 tablet_replica.cc:439] T 9f6c2de37ca943f7860133079d1afece P c4818c2f05694f8abb7074eee8f53398: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5ce38422a138438180879808ba4a2850" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } } }
I20260924 21:12:45.667956 15013 tablet_replica.cc:442] T 9f6c2de37ca943f7860133079d1afece P c4818c2f05694f8abb7074eee8f53398: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:12:45.672327 14991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.670116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:45579 (local address 127.6.210.130:37161)
0924 21:12:45.670161 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:45.670176 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:45.670185 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:12:45.670250 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:45.670251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:45.670286 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:45.670341 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:12:45.670442 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.670893 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.671628 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.671844 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.671920 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.672088 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:12:45.672120 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:12:45.672188 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:45.672247 (+ 59us) server_negotiation.cc:299] Negotiation successful
0924 21:12:45.672265 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 21:12:45.672358 14973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.670051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:37161 (local address 127.6.210.131:45579)
0924 21:12:45.670119 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:45.670130 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:45.670180 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:45.670344 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:45.670344 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:45.670345 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 21:12:45.670387 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.670389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.670971 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.670972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.671544 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.671546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.671918 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.671919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.671974 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.671988 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 21:12:45.672188 (+ 200us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 21:12:45.672190 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 21:12:45.672242 (+ 52us) client_negotiation.cc:240] Negotiation successful
0924 21:12:45.672289 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260924 21:12:45.674398 14781 raft_consensus.cc:1261] T 7d0daeaf270740bd88f41040660798d7 P 5ce38422a138438180879808ba4a2850 [term 1 FOLLOWER]: Refusing update from remote peer c4818c2f05694f8abb7074eee8f53398: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:45.674430 14641 raft_consensus.cc:1261] T 7d0daeaf270740bd88f41040660798d7 P 6e7c05fe428048fdad95f46423ee18c1 [term 1 FOLLOWER]: Refusing update from remote peer c4818c2f05694f8abb7074eee8f53398: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:45.674602 15005 consensus_queue.cc:1035] T 7d0daeaf270740bd88f41040660798d7 P c4818c2f05694f8abb7074eee8f53398 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:12:45.674682 15013 consensus_queue.cc:1035] T 7d0daeaf270740bd88f41040660798d7 P c4818c2f05694f8abb7074eee8f53398 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:12:45.683864 15009 mvcc.cc:205] Tried to move back new op lower bound from 7333004761800777728 to 7333004761335963648. Current Snapshot: MvccSnapshot[applied={T|T < 7333004761800777728}]
I20260924 21:12:45.686223 15038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.682995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:35921 (local address 127.6.210.131:35189)
0924 21:12:45.683036 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:45.683039 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:45.683270 (+ 231us) server_negotiation.cc:407] Connection header received
0924 21:12:45.683453 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:45.683454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:45.683466 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:45.683501 (+ 35us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:12:45.683765 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.684522 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.685384 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.685622 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.685717 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.685936 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:12:45.685979 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:12:45.686045 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:45.686113 (+ 68us) server_negotiation.cc:299] Negotiation successful
0924 21:12:45.686141 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 21:12:45.686312 15051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.682946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:35189 (local address 127.6.210.130:35921)
0924 21:12:45.683197 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:45.683211 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:45.683300 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:45.683582 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:45.683583 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:45.683584 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 21:12:45.683664 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.683666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.684604 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.684605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.685287 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.685289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.685749 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.685751 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:45.685814 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.685829 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 21:12:45.686047 (+ 218us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 21:12:45.686050 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:12:45.686120 (+ 70us) client_negotiation.cc:240] Negotiation successful
0924 21:12:45.686163 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":100,"threads_started":1}
I20260924 21:12:45.695405 14483 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:12:45.695508 14483 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:12:45.696247 14483 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e818eabd67334a1fab61630f7f4583e6: Sending DeleteTablet for 3 replicas of tablet 7d0daeaf270740bd88f41040660798d7
I20260924 21:12:45.696520 14621 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d0daeaf270740bd88f41040660798d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:45 UTC) from {username='oryx', principal='oryx/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:40164
I20260924 21:12:45.696520 14901 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d0daeaf270740bd88f41040660798d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:45 UTC) from {username='oryx', principal='oryx/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:34474
I20260924 21:12:45.696538 14761 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d0daeaf270740bd88f41040660798d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:45 UTC) from {username='oryx', principal='oryx/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:57794
I20260924 21:12:45.696619 15018 tablet_replica.cc:330] T 7d0daeaf270740bd88f41040660798d7 P c4818c2f05694f8abb7074eee8f53398: stopping tablet replica
I20260924 21:12:45.696628 15019 tablet_replica.cc:330] T 7d0daeaf270740bd88f41040660798d7 P 6e7c05fe428048fdad95f46423ee18c1: stopping tablet replica
I20260924 21:12:45.696633 15020 tablet_replica.cc:330] T 7d0daeaf270740bd88f41040660798d7 P 5ce38422a138438180879808ba4a2850: stopping tablet replica
I20260924 21:12:45.696672 15019 raft_consensus.cc:2229] T 7d0daeaf270740bd88f41040660798d7 P 6e7c05fe428048fdad95f46423ee18c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:12:45.696672 15018 raft_consensus.cc:2229] T 7d0daeaf270740bd88f41040660798d7 P c4818c2f05694f8abb7074eee8f53398 [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:12:45.696679 15020 raft_consensus.cc:2229] T 7d0daeaf270740bd88f41040660798d7 P 5ce38422a138438180879808ba4a2850 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:12:45.696714 15020 raft_consensus.cc:2258] T 7d0daeaf270740bd88f41040660798d7 P 5ce38422a138438180879808ba4a2850 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:45.696710 15019 raft_consensus.cc:2258] T 7d0daeaf270740bd88f41040660798d7 P 6e7c05fe428048fdad95f46423ee18c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:45.696749 15018 raft_consensus.cc:2258] T 7d0daeaf270740bd88f41040660798d7 P c4818c2f05694f8abb7074eee8f53398 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:45.697084 15018 ts_tablet_manager.cc:1905] T 7d0daeaf270740bd88f41040660798d7 P c4818c2f05694f8abb7074eee8f53398: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:45.697415 15020 ts_tablet_manager.cc:1905] T 7d0daeaf270740bd88f41040660798d7 P 5ce38422a138438180879808ba4a2850: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:45.699064 15020 ts_tablet_manager.cc:1918] T 7d0daeaf270740bd88f41040660798d7 P 5ce38422a138438180879808ba4a2850: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 21:12:45.699122 15020 log.cc:1196] T 7d0daeaf270740bd88f41040660798d7 P 5ce38422a138438180879808ba4a2850: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-1/wal/wals/7d0daeaf270740bd88f41040660798d7
I20260924 21:12:45.699251 15018 ts_tablet_manager.cc:1918] T 7d0daeaf270740bd88f41040660798d7 P c4818c2f05694f8abb7074eee8f53398: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 21:12:45.699313 15018 log.cc:1196] T 7d0daeaf270740bd88f41040660798d7 P c4818c2f05694f8abb7074eee8f53398: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-2/wal/wals/7d0daeaf270740bd88f41040660798d7
I20260924 21:12:45.699344 15020 ts_tablet_manager.cc:1939] T 7d0daeaf270740bd88f41040660798d7 P 5ce38422a138438180879808ba4a2850: Deleting consensus metadata
I20260924 21:12:45.699566 15018 ts_tablet_manager.cc:1939] T 7d0daeaf270740bd88f41040660798d7 P c4818c2f05694f8abb7074eee8f53398: Deleting consensus metadata
I20260924 21:12:45.699637 14470 catalog_manager.cc:4841] TS 5ce38422a138438180879808ba4a2850 (127.6.210.130:37161): tablet 7d0daeaf270740bd88f41040660798d7 (table test-table [id=bca498d6da5046c391e9fc00d2f7695c]) successfully deleted
I20260924 21:12:45.700037 15019 ts_tablet_manager.cc:1905] T 7d0daeaf270740bd88f41040660798d7 P 6e7c05fe428048fdad95f46423ee18c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:45.700212 14469 catalog_manager.cc:4841] TS c4818c2f05694f8abb7074eee8f53398 (127.6.210.131:35189): tablet 7d0daeaf270740bd88f41040660798d7 (table test-table [id=bca498d6da5046c391e9fc00d2f7695c]) successfully deleted
I20260924 21:12:45.701308 15019 ts_tablet_manager.cc:1918] T 7d0daeaf270740bd88f41040660798d7 P 6e7c05fe428048fdad95f46423ee18c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 21:12:45.701395 15019 log.cc:1196] T 7d0daeaf270740bd88f41040660798d7 P 6e7c05fe428048fdad95f46423ee18c1: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-0/wal/wals/7d0daeaf270740bd88f41040660798d7
I20260924 21:12:45.701613 15019 ts_tablet_manager.cc:1939] T 7d0daeaf270740bd88f41040660798d7 P 6e7c05fe428048fdad95f46423ee18c1: Deleting consensus metadata
I20260924 21:12:45.701954 14470 catalog_manager.cc:4841] TS 6e7c05fe428048fdad95f46423ee18c1 (127.6.210.129:42319): tablet 7d0daeaf270740bd88f41040660798d7 (table test-table [id=bca498d6da5046c391e9fc00d2f7695c]) successfully deleted
Sep 24 21:12:45 dist-test-slave-q276 krb5kdc[14439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 21:12:45 dist-test-slave-q276 krb5kdc[14439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:45 dist-test-slave-q276 krb5kdc[14439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
W20260924 21:12:45.725131 14553 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:45.719304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35568 (local address 127.6.210.190:34811)
0924 21:12:45.719369 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:45.719371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:45.719478 (+ 107us) server_negotiation.cc:407] Connection header received
0924 21:12:45.719547 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:45.719548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:45.719560 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:45.719617 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:45.720170 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.720945 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.721695 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.721998 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.722069 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.725064 (+ 2995us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35568: BlockingRecv error: recv got EOF from 127.0.0.1:35568 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 24 21:12:45 dist-test-slave-q276 krb5kdc[14439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM
I20260924 21:12:45.733631 14553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.726012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35582 (local address 127.6.210.190:34811)
0924 21:12:45.726049 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:45.726050 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:45.726307 (+ 257us) server_negotiation.cc:407] Connection header received
0924 21:12:45.726381 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:45.726382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:45.726391 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:45.726439 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:45.726995 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.727476 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.728487 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.728676 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.728731 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.731077 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:45.731079 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:45.731079 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:45.731086 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:45.732461 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:45.732844 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:45.732846 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:45.732846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:45.732905 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:45.733101 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:45.733102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:45.733102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:45.733238 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:45.733341 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:45.733457 (+ 116us) server_negotiation.cc:299] Negotiation successful
0924 21:12:45.733528 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 21:12:45.735277 14483 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35582:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:12:45.735443 14483 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:12:45.736797 14761 tablet_service.cc:1467] Processing CreateTablet for tablet a953751bb7164e48b4701dc234d0d921 (DEFAULT_TABLE table=test-table [id=50878b82254f485b99545ab6ec41abb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:45.736806 14621 tablet_service.cc:1467] Processing CreateTablet for tablet a953751bb7164e48b4701dc234d0d921 (DEFAULT_TABLE table=test-table [id=50878b82254f485b99545ab6ec41abb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:45.736841 14901 tablet_service.cc:1467] Processing CreateTablet for tablet a953751bb7164e48b4701dc234d0d921 (DEFAULT_TABLE table=test-table [id=50878b82254f485b99545ab6ec41abb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:45.736930 14761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a953751bb7164e48b4701dc234d0d921. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:45.736935 14621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a953751bb7164e48b4701dc234d0d921. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:45.736964 14901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a953751bb7164e48b4701dc234d0d921. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:45.738183 14997 tablet_bootstrap.cc:492] T a953751bb7164e48b4701dc234d0d921 P 6e7c05fe428048fdad95f46423ee18c1: Bootstrap starting.
I20260924 21:12:45.738296 14998 tablet_bootstrap.cc:492] T a953751bb7164e48b4701dc234d0d921 P c4818c2f05694f8abb7074eee8f53398: Bootstrap starting.
I20260924 21:12:45.738637 14998 tablet_bootstrap.cc:654] T a953751bb7164e48b4701dc234d0d921 P c4818c2f05694f8abb7074eee8f53398: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:45.738965 14997 tablet_bootstrap.cc:654] T a953751bb7164e48b4701dc234d0d921 P 6e7c05fe428048fdad95f46423ee18c1: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:45.739349 14995 tablet_bootstrap.cc:492] T a953751bb7164e48b4701dc234d0d921 P 5ce38422a138438180879808ba4a2850: Bootstrap starting.
I20260924 21:12:45.739521 14998 tablet_bootstrap.cc:492] T a953751bb7164e48b4701dc234d0d921 P c4818c2f05694f8abb7074eee8f53398: No bootstrap required, opened a new log
I20260924 21:12:45.739578 14998 ts_tablet_manager.cc:1397] T a953751bb7164e48b4701dc234d0d921 P c4818c2f05694f8abb7074eee8f53398: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:45.739692 14998 raft_consensus.cc:348] T a953751bb7164e48b4701dc234d0d921 P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.739760 14998 raft_consensus.cc:374] T a953751bb7164e48b4701dc234d0d921 P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:45.739751 14995 tablet_bootstrap.cc:654] T a953751bb7164e48b4701dc234d0d921 P 5ce38422a138438180879808ba4a2850: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:45.739781 14998 raft_consensus.cc:729] T a953751bb7164e48b4701dc234d0d921 P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4818c2f05694f8abb7074eee8f53398, State: Initialized, Role: FOLLOWER
I20260924 21:12:45.739823 14998 consensus_queue.cc:260] T a953751bb7164e48b4701dc234d0d921 P c4818c2f05694f8abb7074eee8f53398 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.739908 14998 ts_tablet_manager.cc:1428] T a953751bb7164e48b4701dc234d0d921 P c4818c2f05694f8abb7074eee8f53398: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 21:12:45.740334 14997 tablet_bootstrap.cc:492] T a953751bb7164e48b4701dc234d0d921 P 6e7c05fe428048fdad95f46423ee18c1: No bootstrap required, opened a new log
I20260924 21:12:45.740386 14997 ts_tablet_manager.cc:1397] T a953751bb7164e48b4701dc234d0d921 P 6e7c05fe428048fdad95f46423ee18c1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:12:45.740538 14997 raft_consensus.cc:348] T a953751bb7164e48b4701dc234d0d921 P 6e7c05fe428048fdad95f46423ee18c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.740654 14997 raft_consensus.cc:374] T a953751bb7164e48b4701dc234d0d921 P 6e7c05fe428048fdad95f46423ee18c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:45.740681 14997 raft_consensus.cc:729] T a953751bb7164e48b4701dc234d0d921 P 6e7c05fe428048fdad95f46423ee18c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e7c05fe428048fdad95f46423ee18c1, State: Initialized, Role: FOLLOWER
I20260924 21:12:45.740736 14997 consensus_queue.cc:260] T a953751bb7164e48b4701dc234d0d921 P 6e7c05fe428048fdad95f46423ee18c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.740823 14997 ts_tablet_manager.cc:1428] T a953751bb7164e48b4701dc234d0d921 P 6e7c05fe428048fdad95f46423ee18c1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 21:12:45.741106 14995 tablet_bootstrap.cc:492] T a953751bb7164e48b4701dc234d0d921 P 5ce38422a138438180879808ba4a2850: No bootstrap required, opened a new log
I20260924 21:12:45.741153 14995 ts_tablet_manager.cc:1397] T a953751bb7164e48b4701dc234d0d921 P 5ce38422a138438180879808ba4a2850: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:12:45.741246 14995 raft_consensus.cc:348] T a953751bb7164e48b4701dc234d0d921 P 5ce38422a138438180879808ba4a2850 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.741321 14995 raft_consensus.cc:374] T a953751bb7164e48b4701dc234d0d921 P 5ce38422a138438180879808ba4a2850 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:45.741379 14995 raft_consensus.cc:729] T a953751bb7164e48b4701dc234d0d921 P 5ce38422a138438180879808ba4a2850 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce38422a138438180879808ba4a2850, State: Initialized, Role: FOLLOWER
I20260924 21:12:45.741436 14995 consensus_queue.cc:260] T a953751bb7164e48b4701dc234d0d921 P 5ce38422a138438180879808ba4a2850 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.741530 14995 ts_tablet_manager.cc:1428] T a953751bb7164e48b4701dc234d0d921 P 5ce38422a138438180879808ba4a2850: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:45.907719 15013 raft_consensus.cc:482] T a953751bb7164e48b4701dc234d0d921 P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:12:45.907816 15013 raft_consensus.cc:504] T a953751bb7164e48b4701dc234d0d921 P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.907935 15013 leader_election.cc:288] T a953751bb7164e48b4701dc234d0d921 P c4818c2f05694f8abb7074eee8f53398 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e7c05fe428048fdad95f46423ee18c1 (127.6.210.129:42319), 5ce38422a138438180879808ba4a2850 (127.6.210.130:37161)
I20260924 21:12:45.908187 14641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a953751bb7164e48b4701dc234d0d921" candidate_uuid: "c4818c2f05694f8abb7074eee8f53398" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e7c05fe428048fdad95f46423ee18c1" is_pre_election: true
I20260924 21:12:45.908213 14781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a953751bb7164e48b4701dc234d0d921" candidate_uuid: "c4818c2f05694f8abb7074eee8f53398" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce38422a138438180879808ba4a2850" is_pre_election: true
I20260924 21:12:45.908299 14781 raft_consensus.cc:2454] T a953751bb7164e48b4701dc234d0d921 P 5ce38422a138438180879808ba4a2850 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4818c2f05694f8abb7074eee8f53398 in term 0.
I20260924 21:12:45.908303 14641 raft_consensus.cc:2454] T a953751bb7164e48b4701dc234d0d921 P 6e7c05fe428048fdad95f46423ee18c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4818c2f05694f8abb7074eee8f53398 in term 0.
I20260924 21:12:45.908468 14854 leader_election.cc:302] T a953751bb7164e48b4701dc234d0d921 P c4818c2f05694f8abb7074eee8f53398 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce38422a138438180879808ba4a2850, c4818c2f05694f8abb7074eee8f53398; no voters:
I20260924 21:12:45.908556 15013 raft_consensus.cc:2781] T a953751bb7164e48b4701dc234d0d921 P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:12:45.908599 15013 raft_consensus.cc:482] T a953751bb7164e48b4701dc234d0d921 P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:12:45.908619 15013 raft_consensus.cc:3037] T a953751bb7164e48b4701dc234d0d921 P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:45.909250 15013 raft_consensus.cc:504] T a953751bb7164e48b4701dc234d0d921 P c4818c2f05694f8abb7074eee8f53398 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.909379 15013 leader_election.cc:288] T a953751bb7164e48b4701dc234d0d921 P c4818c2f05694f8abb7074eee8f53398 [CANDIDATE]: Term 1 election: Requested vote from peers 6e7c05fe428048fdad95f46423ee18c1 (127.6.210.129:42319), 5ce38422a138438180879808ba4a2850 (127.6.210.130:37161)
I20260924 21:12:45.909526 14641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a953751bb7164e48b4701dc234d0d921" candidate_uuid: "c4818c2f05694f8abb7074eee8f53398" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e7c05fe428048fdad95f46423ee18c1"
I20260924 21:12:45.909602 14781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a953751bb7164e48b4701dc234d0d921" candidate_uuid: "c4818c2f05694f8abb7074eee8f53398" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce38422a138438180879808ba4a2850"
I20260924 21:12:45.909662 14781 raft_consensus.cc:3037] T a953751bb7164e48b4701dc234d0d921 P 5ce38422a138438180879808ba4a2850 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:45.909655 14641 raft_consensus.cc:3037] T a953751bb7164e48b4701dc234d0d921 P 6e7c05fe428048fdad95f46423ee18c1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:45.910249 14781 raft_consensus.cc:2454] T a953751bb7164e48b4701dc234d0d921 P 5ce38422a138438180879808ba4a2850 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4818c2f05694f8abb7074eee8f53398 in term 1.
I20260924 21:12:45.910468 14854 leader_election.cc:302] T a953751bb7164e48b4701dc234d0d921 P c4818c2f05694f8abb7074eee8f53398 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce38422a138438180879808ba4a2850, c4818c2f05694f8abb7074eee8f53398; no voters:
I20260924 21:12:45.910571 15013 raft_consensus.cc:2781] T a953751bb7164e48b4701dc234d0d921 P c4818c2f05694f8abb7074eee8f53398 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:45.910586 14641 raft_consensus.cc:2454] T a953751bb7164e48b4701dc234d0d921 P 6e7c05fe428048fdad95f46423ee18c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4818c2f05694f8abb7074eee8f53398 in term 1.
I20260924 21:12:45.910646 15013 raft_consensus.cc:686] T a953751bb7164e48b4701dc234d0d921 P c4818c2f05694f8abb7074eee8f53398 [term 1 LEADER]: Becoming Leader. State: Replica: c4818c2f05694f8abb7074eee8f53398, State: Running, Role: LEADER
I20260924 21:12:45.910703 15013 consensus_queue.cc:237] T a953751bb7164e48b4701dc234d0d921 P c4818c2f05694f8abb7074eee8f53398 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.911350 14483 catalog_manager.cc:5495] T a953751bb7164e48b4701dc234d0d921 P c4818c2f05694f8abb7074eee8f53398 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4818c2f05694f8abb7074eee8f53398 (127.6.210.131). New cstate: current_term: 1 leader_uuid: "c4818c2f05694f8abb7074eee8f53398" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } health_report { overall_health: HEALTHY } } }
I20260924 21:12:45.924139 15038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.921889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34482 (local address 127.6.210.131:35189)
0924 21:12:45.921940 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:45.921942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:45.921953 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:12:45.921998 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:45.921999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:45.922010 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:45.922058 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:12:45.922153 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.922733 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.923435 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.923638 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.923714 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.923849 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:12:45.923914 (+ 65us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:12:45.923967 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:45.924016 (+ 49us) server_negotiation.cc:299] Negotiation successful
0924 21:12:45.924034 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 21:12:45.924957 14641 raft_consensus.cc:1261] T a953751bb7164e48b4701dc234d0d921 P 6e7c05fe428048fdad95f46423ee18c1 [term 1 FOLLOWER]: Refusing update from remote peer c4818c2f05694f8abb7074eee8f53398: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:45.924984 14781 raft_consensus.cc:1261] T a953751bb7164e48b4701dc234d0d921 P 5ce38422a138438180879808ba4a2850 [term 1 FOLLOWER]: Refusing update from remote peer c4818c2f05694f8abb7074eee8f53398: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:45.925217 15005 consensus_queue.cc:1035] T a953751bb7164e48b4701dc234d0d921 P c4818c2f05694f8abb7074eee8f53398 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:12:45.925316 15013 consensus_queue.cc:1035] T a953751bb7164e48b4701dc234d0d921 P c4818c2f05694f8abb7074eee8f53398 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:12:45.927690 14483 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:12:45.927772 14483 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:12:45.928400 14483 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e818eabd67334a1fab61630f7f4583e6: Sending DeleteTablet for 3 replicas of tablet a953751bb7164e48b4701dc234d0d921
I20260924 21:12:45.928606 14621 tablet_service.cc:1514] Processing DeleteTablet for tablet a953751bb7164e48b4701dc234d0d921 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:45 UTC) from {username='oryx', principal='oryx/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:40164
I20260924 21:12:45.928661 14761 tablet_service.cc:1514] Processing DeleteTablet for tablet a953751bb7164e48b4701dc234d0d921 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:45 UTC) from {username='oryx', principal='oryx/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:57794
I20260924 21:12:45.928681 14901 tablet_service.cc:1514] Processing DeleteTablet for tablet a953751bb7164e48b4701dc234d0d921 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:45 UTC) from {username='oryx', principal='oryx/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:34474
I20260924 21:12:45.928694 15019 tablet_replica.cc:330] T a953751bb7164e48b4701dc234d0d921 P 6e7c05fe428048fdad95f46423ee18c1: stopping tablet replica
I20260924 21:12:45.928754 15019 raft_consensus.cc:2229] T a953751bb7164e48b4701dc234d0d921 P 6e7c05fe428048fdad95f46423ee18c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:12:45.928758 15020 tablet_replica.cc:330] T a953751bb7164e48b4701dc234d0d921 P 5ce38422a138438180879808ba4a2850: stopping tablet replica
I20260924 21:12:45.928761 15018 tablet_replica.cc:330] T a953751bb7164e48b4701dc234d0d921 P c4818c2f05694f8abb7074eee8f53398: stopping tablet replica
I20260924 21:12:45.928808 15018 raft_consensus.cc:2229] T a953751bb7164e48b4701dc234d0d921 P c4818c2f05694f8abb7074eee8f53398 [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:12:45.928820 15019 raft_consensus.cc:2258] T a953751bb7164e48b4701dc234d0d921 P 6e7c05fe428048fdad95f46423ee18c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:45.928823 15020 raft_consensus.cc:2229] T a953751bb7164e48b4701dc234d0d921 P 5ce38422a138438180879808ba4a2850 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:12:45.928867 15018 raft_consensus.cc:2258] T a953751bb7164e48b4701dc234d0d921 P c4818c2f05694f8abb7074eee8f53398 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:45.928876 15020 raft_consensus.cc:2258] T a953751bb7164e48b4701dc234d0d921 P 5ce38422a138438180879808ba4a2850 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:45.929168 15018 ts_tablet_manager.cc:1905] T a953751bb7164e48b4701dc234d0d921 P c4818c2f05694f8abb7074eee8f53398: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:45.929306 15020 ts_tablet_manager.cc:1905] T a953751bb7164e48b4701dc234d0d921 P 5ce38422a138438180879808ba4a2850: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:45.929306 15019 ts_tablet_manager.cc:1905] T a953751bb7164e48b4701dc234d0d921 P 6e7c05fe428048fdad95f46423ee18c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:45.930523 15018 ts_tablet_manager.cc:1918] T a953751bb7164e48b4701dc234d0d921 P c4818c2f05694f8abb7074eee8f53398: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:12:45.930593 15018 log.cc:1196] T a953751bb7164e48b4701dc234d0d921 P c4818c2f05694f8abb7074eee8f53398: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-2/wal/wals/a953751bb7164e48b4701dc234d0d921
I20260924 21:12:45.930792 15018 ts_tablet_manager.cc:1939] T a953751bb7164e48b4701dc234d0d921 P c4818c2f05694f8abb7074eee8f53398: Deleting consensus metadata
I20260924 21:12:45.930826 15019 ts_tablet_manager.cc:1918] T a953751bb7164e48b4701dc234d0d921 P 6e7c05fe428048fdad95f46423ee18c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:12:45.930902 15019 log.cc:1196] T a953751bb7164e48b4701dc234d0d921 P 6e7c05fe428048fdad95f46423ee18c1: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-0/wal/wals/a953751bb7164e48b4701dc234d0d921
I20260924 21:12:45.930948 15020 ts_tablet_manager.cc:1918] T a953751bb7164e48b4701dc234d0d921 P 5ce38422a138438180879808ba4a2850: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:12:45.931010 15020 log.cc:1196] T a953751bb7164e48b4701dc234d0d921 P 5ce38422a138438180879808ba4a2850: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-1/wal/wals/a953751bb7164e48b4701dc234d0d921
I20260924 21:12:45.931110 14469 catalog_manager.cc:4841] TS c4818c2f05694f8abb7074eee8f53398 (127.6.210.131:35189): tablet a953751bb7164e48b4701dc234d0d921 (table test-table [id=50878b82254f485b99545ab6ec41abb2]) successfully deleted
I20260924 21:12:45.931160 15019 ts_tablet_manager.cc:1939] T a953751bb7164e48b4701dc234d0d921 P 6e7c05fe428048fdad95f46423ee18c1: Deleting consensus metadata
I20260924 21:12:45.931250 15020 ts_tablet_manager.cc:1939] T a953751bb7164e48b4701dc234d0d921 P 5ce38422a138438180879808ba4a2850: Deleting consensus metadata
I20260924 21:12:45.931479 14470 catalog_manager.cc:4841] TS 6e7c05fe428048fdad95f46423ee18c1 (127.6.210.129:42319): tablet a953751bb7164e48b4701dc234d0d921 (table test-table [id=50878b82254f485b99545ab6ec41abb2]) successfully deleted
I20260924 21:12:45.931631 14470 catalog_manager.cc:4841] TS 5ce38422a138438180879808ba4a2850 (127.6.210.130:37161): tablet a953751bb7164e48b4701dc234d0d921 (table test-table [id=50878b82254f485b99545ab6ec41abb2]) successfully deleted
I20260924 21:12:45.931689 14483 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35582:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:12:45.931816 14483 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:12:45.933101 14761 tablet_service.cc:1467] Processing CreateTablet for tablet 8f16dca7bada40cca347260d7eb4c273 (DEFAULT_TABLE table=test-table [id=7c2c3ecb33fc429299eec76da41e5cd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:45.933116 14621 tablet_service.cc:1467] Processing CreateTablet for tablet 8f16dca7bada40cca347260d7eb4c273 (DEFAULT_TABLE table=test-table [id=7c2c3ecb33fc429299eec76da41e5cd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:45.933151 14901 tablet_service.cc:1467] Processing CreateTablet for tablet 8f16dca7bada40cca347260d7eb4c273 (DEFAULT_TABLE table=test-table [id=7c2c3ecb33fc429299eec76da41e5cd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:12:45.933220 14761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f16dca7bada40cca347260d7eb4c273. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:45.933233 14901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f16dca7bada40cca347260d7eb4c273. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:45.933238 14621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f16dca7bada40cca347260d7eb4c273. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:45.934106 14995 tablet_bootstrap.cc:492] T 8f16dca7bada40cca347260d7eb4c273 P 5ce38422a138438180879808ba4a2850: Bootstrap starting.
I20260924 21:12:45.934369 14997 tablet_bootstrap.cc:492] T 8f16dca7bada40cca347260d7eb4c273 P 6e7c05fe428048fdad95f46423ee18c1: Bootstrap starting.
I20260924 21:12:45.934456 14995 tablet_bootstrap.cc:654] T 8f16dca7bada40cca347260d7eb4c273 P 5ce38422a138438180879808ba4a2850: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:45.934526 14998 tablet_bootstrap.cc:492] T 8f16dca7bada40cca347260d7eb4c273 P c4818c2f05694f8abb7074eee8f53398: Bootstrap starting.
I20260924 21:12:45.934778 14997 tablet_bootstrap.cc:654] T 8f16dca7bada40cca347260d7eb4c273 P 6e7c05fe428048fdad95f46423ee18c1: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:45.934854 14998 tablet_bootstrap.cc:654] T 8f16dca7bada40cca347260d7eb4c273 P c4818c2f05694f8abb7074eee8f53398: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:45.935014 14995 tablet_bootstrap.cc:492] T 8f16dca7bada40cca347260d7eb4c273 P 5ce38422a138438180879808ba4a2850: No bootstrap required, opened a new log
I20260924 21:12:45.935086 14995 ts_tablet_manager.cc:1397] T 8f16dca7bada40cca347260d7eb4c273 P 5ce38422a138438180879808ba4a2850: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:45.935211 14995 raft_consensus.cc:348] T 8f16dca7bada40cca347260d7eb4c273 P 5ce38422a138438180879808ba4a2850 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.935308 14995 raft_consensus.cc:374] T 8f16dca7bada40cca347260d7eb4c273 P 5ce38422a138438180879808ba4a2850 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:45.935340 14995 raft_consensus.cc:729] T 8f16dca7bada40cca347260d7eb4c273 P 5ce38422a138438180879808ba4a2850 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce38422a138438180879808ba4a2850, State: Initialized, Role: FOLLOWER
I20260924 21:12:45.935392 14995 consensus_queue.cc:260] T 8f16dca7bada40cca347260d7eb4c273 P 5ce38422a138438180879808ba4a2850 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.935519 14995 ts_tablet_manager.cc:1428] T 8f16dca7bada40cca347260d7eb4c273 P 5ce38422a138438180879808ba4a2850: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 21:12:45.935580 14998 tablet_bootstrap.cc:492] T 8f16dca7bada40cca347260d7eb4c273 P c4818c2f05694f8abb7074eee8f53398: No bootstrap required, opened a new log
I20260924 21:12:45.935629 14998 ts_tablet_manager.cc:1397] T 8f16dca7bada40cca347260d7eb4c273 P c4818c2f05694f8abb7074eee8f53398: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:45.935731 14998 raft_consensus.cc:348] T 8f16dca7bada40cca347260d7eb4c273 P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.935796 14998 raft_consensus.cc:374] T 8f16dca7bada40cca347260d7eb4c273 P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:45.935817 14998 raft_consensus.cc:729] T 8f16dca7bada40cca347260d7eb4c273 P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4818c2f05694f8abb7074eee8f53398, State: Initialized, Role: FOLLOWER
I20260924 21:12:45.935859 14998 consensus_queue.cc:260] T 8f16dca7bada40cca347260d7eb4c273 P c4818c2f05694f8abb7074eee8f53398 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.935942 14998 ts_tablet_manager.cc:1428] T 8f16dca7bada40cca347260d7eb4c273 P c4818c2f05694f8abb7074eee8f53398: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:45.939215 14997 tablet_bootstrap.cc:492] T 8f16dca7bada40cca347260d7eb4c273 P 6e7c05fe428048fdad95f46423ee18c1: No bootstrap required, opened a new log
I20260924 21:12:45.939303 14997 ts_tablet_manager.cc:1397] T 8f16dca7bada40cca347260d7eb4c273 P 6e7c05fe428048fdad95f46423ee18c1: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260924 21:12:45.939440 14997 raft_consensus.cc:348] T 8f16dca7bada40cca347260d7eb4c273 P 6e7c05fe428048fdad95f46423ee18c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.939512 14997 raft_consensus.cc:374] T 8f16dca7bada40cca347260d7eb4c273 P 6e7c05fe428048fdad95f46423ee18c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:45.939543 14997 raft_consensus.cc:729] T 8f16dca7bada40cca347260d7eb4c273 P 6e7c05fe428048fdad95f46423ee18c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e7c05fe428048fdad95f46423ee18c1, State: Initialized, Role: FOLLOWER
I20260924 21:12:45.939592 14997 consensus_queue.cc:260] T 8f16dca7bada40cca347260d7eb4c273 P 6e7c05fe428048fdad95f46423ee18c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.939695 14997 ts_tablet_manager.cc:1428] T 8f16dca7bada40cca347260d7eb4c273 P 6e7c05fe428048fdad95f46423ee18c1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:45.975854 15039 raft_consensus.cc:482] T 8f16dca7bada40cca347260d7eb4c273 P 5ce38422a138438180879808ba4a2850 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:12:45.975957 15039 raft_consensus.cc:504] T 8f16dca7bada40cca347260d7eb4c273 P 5ce38422a138438180879808ba4a2850 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.976163 15039 leader_election.cc:288] T 8f16dca7bada40cca347260d7eb4c273 P 5ce38422a138438180879808ba4a2850 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e7c05fe428048fdad95f46423ee18c1 (127.6.210.129:42319), c4818c2f05694f8abb7074eee8f53398 (127.6.210.131:35189)
I20260924 21:12:45.976377 14921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f16dca7bada40cca347260d7eb4c273" candidate_uuid: "5ce38422a138438180879808ba4a2850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4818c2f05694f8abb7074eee8f53398" is_pre_election: true
I20260924 21:12:45.976384 14641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f16dca7bada40cca347260d7eb4c273" candidate_uuid: "5ce38422a138438180879808ba4a2850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e7c05fe428048fdad95f46423ee18c1" is_pre_election: true
I20260924 21:12:45.976485 14641 raft_consensus.cc:2454] T 8f16dca7bada40cca347260d7eb4c273 P 6e7c05fe428048fdad95f46423ee18c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce38422a138438180879808ba4a2850 in term 0.
I20260924 21:12:45.976483 14921 raft_consensus.cc:2454] T 8f16dca7bada40cca347260d7eb4c273 P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce38422a138438180879808ba4a2850 in term 0.
I20260924 21:12:45.976670 14714 leader_election.cc:302] T 8f16dca7bada40cca347260d7eb4c273 P 5ce38422a138438180879808ba4a2850 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce38422a138438180879808ba4a2850, 6e7c05fe428048fdad95f46423ee18c1; no voters:
I20260924 21:12:45.976775 15039 raft_consensus.cc:2781] T 8f16dca7bada40cca347260d7eb4c273 P 5ce38422a138438180879808ba4a2850 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:12:45.976816 15039 raft_consensus.cc:482] T 8f16dca7bada40cca347260d7eb4c273 P 5ce38422a138438180879808ba4a2850 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:12:45.976842 15039 raft_consensus.cc:3037] T 8f16dca7bada40cca347260d7eb4c273 P 5ce38422a138438180879808ba4a2850 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:45.977545 15039 raft_consensus.cc:504] T 8f16dca7bada40cca347260d7eb4c273 P 5ce38422a138438180879808ba4a2850 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.977705 15039 leader_election.cc:288] T 8f16dca7bada40cca347260d7eb4c273 P 5ce38422a138438180879808ba4a2850 [CANDIDATE]: Term 1 election: Requested vote from peers 6e7c05fe428048fdad95f46423ee18c1 (127.6.210.129:42319), c4818c2f05694f8abb7074eee8f53398 (127.6.210.131:35189)
I20260924 21:12:45.977890 14921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f16dca7bada40cca347260d7eb4c273" candidate_uuid: "5ce38422a138438180879808ba4a2850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4818c2f05694f8abb7074eee8f53398"
I20260924 21:12:45.977905 14641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f16dca7bada40cca347260d7eb4c273" candidate_uuid: "5ce38422a138438180879808ba4a2850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e7c05fe428048fdad95f46423ee18c1"
I20260924 21:12:45.977980 14921 raft_consensus.cc:3037] T 8f16dca7bada40cca347260d7eb4c273 P c4818c2f05694f8abb7074eee8f53398 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:45.977988 14641 raft_consensus.cc:3037] T 8f16dca7bada40cca347260d7eb4c273 P 6e7c05fe428048fdad95f46423ee18c1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:45.978807 14641 raft_consensus.cc:2454] T 8f16dca7bada40cca347260d7eb4c273 P 6e7c05fe428048fdad95f46423ee18c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce38422a138438180879808ba4a2850 in term 1.
I20260924 21:12:45.978878 14921 raft_consensus.cc:2454] T 8f16dca7bada40cca347260d7eb4c273 P c4818c2f05694f8abb7074eee8f53398 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce38422a138438180879808ba4a2850 in term 1.
I20260924 21:12:45.979015 14714 leader_election.cc:302] T 8f16dca7bada40cca347260d7eb4c273 P 5ce38422a138438180879808ba4a2850 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce38422a138438180879808ba4a2850, 6e7c05fe428048fdad95f46423ee18c1; no voters:
I20260924 21:12:45.979118 15039 raft_consensus.cc:2781] T 8f16dca7bada40cca347260d7eb4c273 P 5ce38422a138438180879808ba4a2850 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:45.979223 15039 raft_consensus.cc:686] T 8f16dca7bada40cca347260d7eb4c273 P 5ce38422a138438180879808ba4a2850 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce38422a138438180879808ba4a2850, State: Running, Role: LEADER
I20260924 21:12:45.979293 15039 consensus_queue.cc:237] T 8f16dca7bada40cca347260d7eb4c273 P 5ce38422a138438180879808ba4a2850 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } }
I20260924 21:12:45.979851 14484 catalog_manager.cc:5495] T 8f16dca7bada40cca347260d7eb4c273 P 5ce38422a138438180879808ba4a2850 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ce38422a138438180879808ba4a2850 (127.6.210.130). New cstate: current_term: 1 leader_uuid: "5ce38422a138438180879808ba4a2850" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ce38422a138438180879808ba4a2850" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 37161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 } health_report { overall_health: UNKNOWN } } }
I20260924 21:12:45.997545 14991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.995572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57812 (local address 127.6.210.130:37161)
0924 21:12:45.995671 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:45.995673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:45.995686 (+ 13us) server_negotiation.cc:407] Connection header received
0924 21:12:45.995772 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:45.995773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:45.995781 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:45.995822 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:12:45.995908 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.996358 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.997029 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:45.997202 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:45.997252 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:45.997361 (+ 109us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:12:45.997385 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:12:45.997423 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:45.997472 (+ 49us) server_negotiation.cc:299] Negotiation successful
0924 21:12:45.997488 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260924 21:12:46.001843 15051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.999587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:37161 (local address 127.6.210.130:41451)
0924 21:12:45.999679 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:45.999691 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:45.999742 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:45.999866 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:45.999867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:45.999868 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 21:12:45.999901 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:45.999903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:46.000527 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:46.000528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:46.001061 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:46.001063 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:46.001432 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:46.001432 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:46.001478 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:46.001483 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 21:12:46.001656 (+ 173us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 21:12:46.001658 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 21:12:46.001717 (+ 59us) client_negotiation.cc:240] Negotiation successful
0924 21:12:46.001763 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":2}
I20260924 21:12:46.001843 14991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:45.999660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:41451 (local address 127.6.210.130:37161)
0924 21:12:45.999695 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:45.999698 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:45.999725 (+ 27us) server_negotiation.cc:407] Connection header received
0924 21:12:45.999801 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:45.999802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:45.999810 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:45.999859 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:12:45.999947 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:46.000439 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:46.001162 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:46.001368 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:46.001433 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:46.001568 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:12:46.001596 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:12:46.001647 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:46.001722 (+ 75us) server_negotiation.cc:299] Negotiation successful
0924 21:12:46.001739 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":17}
I20260924 21:12:46.003664 14921 raft_consensus.cc:1261] T 8f16dca7bada40cca347260d7eb4c273 P c4818c2f05694f8abb7074eee8f53398 [term 1 FOLLOWER]: Refusing update from remote peer 5ce38422a138438180879808ba4a2850: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:46.003682 14641 raft_consensus.cc:1261] T 8f16dca7bada40cca347260d7eb4c273 P 6e7c05fe428048fdad95f46423ee18c1 [term 1 FOLLOWER]: Refusing update from remote peer 5ce38422a138438180879808ba4a2850: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:12:46.003875 14999 consensus_queue.cc:1035] T 8f16dca7bada40cca347260d7eb4c273 P 5ce38422a138438180879808ba4a2850 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4818c2f05694f8abb7074eee8f53398" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:12:46.004002 15039 consensus_queue.cc:1035] T 8f16dca7bada40cca347260d7eb4c273 P 5ce38422a138438180879808ba4a2850 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e7c05fe428048fdad95f46423ee18c1" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:12:46.014847 14484 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:12:46.014961 14484 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:12:46.015619 14484 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e818eabd67334a1fab61630f7f4583e6: Sending DeleteTablet for 3 replicas of tablet 8f16dca7bada40cca347260d7eb4c273
I20260924 21:12:46.015847 14901 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f16dca7bada40cca347260d7eb4c273 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:46 UTC) from {username='oryx', principal='oryx/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:34474
I20260924 21:12:46.015935 15018 tablet_replica.cc:330] T 8f16dca7bada40cca347260d7eb4c273 P c4818c2f05694f8abb7074eee8f53398: stopping tablet replica
I20260924 21:12:46.015928 14761 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f16dca7bada40cca347260d7eb4c273 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:46 UTC) from {username='oryx', principal='oryx/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:57794
I20260924 21:12:46.015978 15018 raft_consensus.cc:2229] T 8f16dca7bada40cca347260d7eb4c273 P c4818c2f05694f8abb7074eee8f53398 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:12:46.016022 15020 tablet_replica.cc:330] T 8f16dca7bada40cca347260d7eb4c273 P 5ce38422a138438180879808ba4a2850: stopping tablet replica
I20260924 21:12:46.016079 15020 raft_consensus.cc:2229] T 8f16dca7bada40cca347260d7eb4c273 P 5ce38422a138438180879808ba4a2850 [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:12:46.016083 15018 raft_consensus.cc:2258] T 8f16dca7bada40cca347260d7eb4c273 P c4818c2f05694f8abb7074eee8f53398 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:46.016168 15020 raft_consensus.cc:2258] T 8f16dca7bada40cca347260d7eb4c273 P 5ce38422a138438180879808ba4a2850 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:46.016227 14621 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f16dca7bada40cca347260d7eb4c273 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:12:46 UTC) from {username='oryx', principal='oryx/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:40164
I20260924 21:12:46.016316 15019 tablet_replica.cc:330] T 8f16dca7bada40cca347260d7eb4c273 P 6e7c05fe428048fdad95f46423ee18c1: stopping tablet replica
I20260924 21:12:46.016379 15019 raft_consensus.cc:2229] T 8f16dca7bada40cca347260d7eb4c273 P 6e7c05fe428048fdad95f46423ee18c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:12:46.016386 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 14556
I20260924 21:12:46.016412 15019 raft_consensus.cc:2258] T 8f16dca7bada40cca347260d7eb4c273 P 6e7c05fe428048fdad95f46423ee18c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:12:46.017184 15020 ts_tablet_manager.cc:1905] T 8f16dca7bada40cca347260d7eb4c273 P 5ce38422a138438180879808ba4a2850: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:46.019006 15020 ts_tablet_manager.cc:1918] T 8f16dca7bada40cca347260d7eb4c273 P 5ce38422a138438180879808ba4a2850: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 21:12:46.019064 15020 log.cc:1196] T 8f16dca7bada40cca347260d7eb4c273 P 5ce38422a138438180879808ba4a2850: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-1/wal/wals/8f16dca7bada40cca347260d7eb4c273
I20260924 21:12:46.019285 15020 ts_tablet_manager.cc:1939] T 8f16dca7bada40cca347260d7eb4c273 P 5ce38422a138438180879808ba4a2850: Deleting consensus metadata
I20260924 21:12:46.019709 15018 ts_tablet_manager.cc:1905] T 8f16dca7bada40cca347260d7eb4c273 P c4818c2f05694f8abb7074eee8f53398: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:12:46.019730 14470 catalog_manager.cc:4841] TS 5ce38422a138438180879808ba4a2850 (127.6.210.130:37161): tablet 8f16dca7bada40cca347260d7eb4c273 (table test-table [id=7c2c3ecb33fc429299eec76da41e5cd5]) successfully deleted
I20260924 21:12:46.021121 15018 ts_tablet_manager.cc:1918] T 8f16dca7bada40cca347260d7eb4c273 P c4818c2f05694f8abb7074eee8f53398: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 21:12:46.021185 15018 log.cc:1196] T 8f16dca7bada40cca347260d7eb4c273 P c4818c2f05694f8abb7074eee8f53398: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284333163253-6986-0/minicluster-data/ts-2/wal/wals/8f16dca7bada40cca347260d7eb4c273
I20260924 21:12:46.021401 15018 ts_tablet_manager.cc:1939] T 8f16dca7bada40cca347260d7eb4c273 P c4818c2f05694f8abb7074eee8f53398: Deleting consensus metadata
I20260924 21:12:46.021785 14469 catalog_manager.cc:4841] TS c4818c2f05694f8abb7074eee8f53398 (127.6.210.131:35189): tablet 8f16dca7bada40cca347260d7eb4c273 (table test-table [id=7c2c3ecb33fc429299eec76da41e5cd5]) successfully deleted
W20260924 21:12:46.024336 14470 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.210.129:42319 (error 108)
W20260924 21:12:46.024904 15082 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:46.024725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:42319 (local address 127.0.0.1:40172)
0924 21:12:46.024816 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:46.024857 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.129:42319: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":74,"thread_start_us":56,"threads_started":1}
W20260924 21:12:46.025002 14470 catalog_manager.cc:4584] TS 6e7c05fe428048fdad95f46423ee18c1 (127.6.210.129:42319): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8f16dca7bada40cca347260d7eb4c273: Network error: Client connection negotiation failed: client connection to 127.6.210.129:42319: connect: Connection reset by peer (error 104)
I20260924 21:12:46.025319 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 14696
I20260924 21:12:46.030470 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 14836
I20260924 21:12:46.036592 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 14452
2026-09-24T21:12:46Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2640 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:12:46 dist-test-slave-q276 krb5kdc[15085](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:12:46 dist-test-slave-q276 krb5kdc[15085](info): set up 2 sockets
Sep 24 21:12:46 dist-test-slave-q276 krb5kdc[15085](info): commencing operation
krb5kdc: starting...
W20260924 21:12:47.075030 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:12:47 dist-test-slave-q276 krb5kdc[15085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:12:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:12:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.210.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
I20260924 21:12:47.192505 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:43125
--webserver_interface=127.6.210.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:42329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:43125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:47.272800 15098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:47.273048 15098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:47.273087 15098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:47.274324 15098 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:12:47.274391 15098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:47.274416 15098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:47.274447 15098 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:12:47.274477 15098 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:12:47.275998 15098 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:42329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:43125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:43125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:47.276293 15098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:47.276531 15098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:47.281450 15104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:47.281463 15106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:47.281720 15098 server_base.cc:1034] running on GCE node
W20260924 21:12:47.281450 15103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:47.282018 15098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:47.282375 15098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:47.283540 15098 hybrid_clock.cc:648] HybridClock initialized: now 1790284367283525 us; error 26 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-24T21:12:47Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1251 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:12:47 dist-test-slave-q276 krb5kdc[15111](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:12:47 dist-test-slave-q276 krb5kdc[15111](info): set up 2 sockets
Sep 24 21:12:47 dist-test-slave-q276 krb5kdc[15111](info): commencing operation
krb5kdc: starting...
W20260924 21:12:48.379675 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:12:48 dist-test-slave-q276 krb5kdc[15111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:12:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:12:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.210.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
I20260924 21:12:48.501387 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:37673
--webserver_interface=127.6.210.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:37673
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:48.581652 15124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:48.581918 15124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:48.581954 15124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:48.583140 15124 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:12:48.583214 15124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:48.583238 15124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:48.583257 15124 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:12:48.583277 15124 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:12:48.584714 15124 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:37673
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:37673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:48.584993 15124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:48.585220 15124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:48.590085 15132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:48.590248 15129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:48.590229 15130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:48.590309 15124 server_base.cc:1034] running on GCE node
I20260924 21:12:48.590696 15124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:48.590988 15124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:48.592159 15124 hybrid_clock.cc:648] HybridClock initialized: now 1790284368592147 us; error 47 us; skew 500 ppm
Sep 24 21:12:48 dist-test-slave-q276 krb5kdc[15111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284368, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:48.594698 15124 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:12:48.595170 15124 webserver.cc:460] Webserver started at http://127.6.210.190:39251/ using document root <none> and password file <none>
I20260924 21:12:48.595371 15124 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:48.595412 15124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:48.595498 15124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:48.596495 15124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "236719135af64bbea3ef77417f3aa47b"
format_stamp: "Formatted at 2026-09-24 21:12:48 on dist-test-slave-q276"
server_key: "3c2b2296b1bfe776fbb01a75adf4922b"
server_key_iv: "ab0d3d5b13b3ac1c9dccd55e78bbe46a"
server_key_version: "encryptionkey@0"
I20260924 21:12:48.596815 15124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "236719135af64bbea3ef77417f3aa47b"
format_stamp: "Formatted at 2026-09-24 21:12:48 on dist-test-slave-q276"
server_key: "3c2b2296b1bfe776fbb01a75adf4922b"
server_key_iv: "ab0d3d5b13b3ac1c9dccd55e78bbe46a"
server_key_version: "encryptionkey@0"
I20260924 21:12:48.598065 15124 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:12:48.598841 15139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:48.599112 15124 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 21:12:48.599197 15124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "236719135af64bbea3ef77417f3aa47b"
format_stamp: "Formatted at 2026-09-24 21:12:48 on dist-test-slave-q276"
server_key: "3c2b2296b1bfe776fbb01a75adf4922b"
server_key_iv: "ab0d3d5b13b3ac1c9dccd55e78bbe46a"
server_key_version: "encryptionkey@0"
I20260924 21:12:48.599255 15124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:48.614337 15124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:48.616364 15124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:48.616480 15124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:48.620585 15124 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:37673
I20260924 21:12:48.620644 15191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:37673 every 8 connection(s)
I20260924 21:12:48.621090 15124 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:12:48.621999 15192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:12:48.624351 15192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b: Bootstrap starting.
I20260924 21:12:48.624926 15192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b: Neither blocks nor log segments found. Creating new log.
I20260924 21:12:48.625208 15192 log.cc:824] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:48.625928 15192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b: No bootstrap required, opened a new log
I20260924 21:12:48.626312 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 15124
I20260924 21:12:48.626430 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:12:48.626639 6986 external_mini_cluster.cc:1428] Setting key 160108bc9b95cd5cd19a305f87deb801
I20260924 21:12:48.628330 15192 raft_consensus.cc:348] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "236719135af64bbea3ef77417f3aa47b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37673 } }
I20260924 21:12:48.628486 15192 raft_consensus.cc:374] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:12:48.628532 15192 raft_consensus.cc:729] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 236719135af64bbea3ef77417f3aa47b, State: Initialized, Role: FOLLOWER
I20260924 21:12:48.628664 15192 consensus_queue.cc:260] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "236719135af64bbea3ef77417f3aa47b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37673 } }
I20260924 21:12:48.628762 15192 raft_consensus.cc:388] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:12:48.628818 15192 raft_consensus.cc:482] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:12:48.628870 15192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:12:48.629400 15192 raft_consensus.cc:504] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "236719135af64bbea3ef77417f3aa47b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37673 } }
I20260924 21:12:48.629485 15192 leader_election.cc:302] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 236719135af64bbea3ef77417f3aa47b; no voters:
I20260924 21:12:48.629668 15192 leader_election.cc:288] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:12:48.629695 15197 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:12:48.629813 15197 raft_consensus.cc:686] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [term 1 LEADER]: Becoming Leader. State: Replica: 236719135af64bbea3ef77417f3aa47b, State: Running, Role: LEADER
I20260924 21:12:48.629894 15197 consensus_queue.cc:237] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "236719135af64bbea3ef77417f3aa47b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37673 } }
I20260924 21:12:48.629923 15192 sys_catalog.cc:564] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:12:48.630211 15197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "236719135af64bbea3ef77417f3aa47b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "236719135af64bbea3ef77417f3aa47b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37673 } } }
I20260924 21:12:48.630321 15197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:48.630424 15198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 236719135af64bbea3ef77417f3aa47b. Latest consensus state: current_term: 1 leader_uuid: "236719135af64bbea3ef77417f3aa47b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "236719135af64bbea3ef77417f3aa47b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37673 } } }
I20260924 21:12:48.630522 15198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:48.630700 15203 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:12:48.631430 15203 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:12:48.633150 15203 catalog_manager.cc:1347] Generated new cluster ID: 4a368ced3174462995ca0025b7222850
I20260924 21:12:48.633220 15203 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 21:12:48 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:48.638823 15196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:48.627262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52282 (local address 127.6.210.190:37673)
0924 21:12:48.627429 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:48.627439 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:48.627470 (+ 31us) server_negotiation.cc:407] Connection header received
0924 21:12:48.627906 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:48.627915 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:48.627980 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:48.628137 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:48.628820 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:48.630476 (+ 1656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:48.631586 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:48.632218 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:48.632323 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:48.635213 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:48.635216 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:48.635217 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:48.635228 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:48.637165 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:48.637614 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:48.637617 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:48.637617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:48.637719 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:48.637932 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:48.637933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:48.637933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:48.638207 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:48.638378 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:48.638505 (+ 127us) server_negotiation.cc:299] Negotiation successful
0924 21:12:48.638632 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":70,"threads_started":1}
I20260924 21:12:48.641397 15203 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:12:48.641894 15203 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:12:48.646895 15203 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b: Generated new TSK 0
I20260924 21:12:48.647079 15203 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.210.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
I20260924 21:12:48.677014 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.129:0
--local_ip_for_outbound_sockets=127.6.210.129
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:37673
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:48.757519 15218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:48.757723 15218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:48.757746 15218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:48.758994 15218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:48.759035 15218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:48.759064 15218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.129
I20260924 21:12:48.760479 15218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:37673
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.129
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:48.760722 15218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:48.760970 15218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 21:12:48.766055 15218 server_base.cc:1034] running on GCE node
W20260924 21:12:48.766038 15223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:48.766023 15226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:48.766028 15224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:48.766500 15218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:48.766767 15218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:48.767931 15218 hybrid_clock.cc:648] HybridClock initialized: now 1790284368767918 us; error 45 us; skew 500 ppm
Sep 24 21:12:48 dist-test-slave-q276 krb5kdc[15111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284368, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:48.770381 15218 init.cc:377] Logged in from keytab as kudu/127.6.210.129@KRBTEST.COM (short username kudu)
I20260924 21:12:48.770857 15218 webserver.cc:460] Webserver started at http://127.6.210.129:40189/ using document root <none> and password file <none>
I20260924 21:12:48.771103 15218 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:48.771173 15218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:48.771306 15218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:48.772344 15218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-0/data/instance:
uuid: "e9a809e392de45bdb29ec0f4bea7d2c7"
format_stamp: "Formatted at 2026-09-24 21:12:48 on dist-test-slave-q276"
server_key: "bba249730873c5d44ab52216545b9072"
server_key_iv: "a3df7496397b7610290ca1cd54cd9996"
server_key_version: "encryptionkey@0"
I20260924 21:12:48.772706 15218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance:
uuid: "e9a809e392de45bdb29ec0f4bea7d2c7"
format_stamp: "Formatted at 2026-09-24 21:12:48 on dist-test-slave-q276"
server_key: "bba249730873c5d44ab52216545b9072"
server_key_iv: "a3df7496397b7610290ca1cd54cd9996"
server_key_version: "encryptionkey@0"
I20260924 21:12:48.774009 15218 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 21:12:48.774847 15233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:48.775048 15218 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 21:12:48.775144 15218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-0/wal
uuid: "e9a809e392de45bdb29ec0f4bea7d2c7"
format_stamp: "Formatted at 2026-09-24 21:12:48 on dist-test-slave-q276"
server_key: "bba249730873c5d44ab52216545b9072"
server_key_iv: "a3df7496397b7610290ca1cd54cd9996"
server_key_version: "encryptionkey@0"
I20260924 21:12:48.775230 15218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:48.793043 15218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:48.795130 15218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:48.795312 15218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:48.795563 15218 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:48.795879 15218 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:48.795935 15218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:48.795976 15218 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:48.796005 15218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:48.802353 15218 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.129:42923
I20260924 21:12:48.802397 15346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.129:42923 every 8 connection(s)
I20260924 21:12:48.802747 15218 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
Sep 24 21:12:48 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284368, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:48.811973 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 15218
I20260924 21:12:48.812088 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance
I20260924 21:12:48.812345 6986 external_mini_cluster.cc:1428] Setting key 918863592259effe609f083c7e71ba58
I20260924 21:12:48.812355 15196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:48.803724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:48473 (local address 127.6.210.190:37673)
0924 21:12:48.803844 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:48.803847 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:48.804018 (+ 171us) server_negotiation.cc:407] Connection header received
0924 21:12:48.804555 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:48.804556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:48.804568 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:48.804621 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:48.805635 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:48.806122 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:48.807196 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:48.807397 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:48.807479 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:48.809670 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:48.809671 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:48.809672 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:48.809680 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:48.811003 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:48.811376 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:48.811377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:48.811377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:48.811427 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:48.811640 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:48.811640 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:48.811641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:48.811758 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:48.811806 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:48.812211 (+ 405us) server_negotiation.cc:299] Negotiation successful
0924 21:12:48.812291 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260924 21:12:48.812661 15349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:48.803652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.129:48473)
0924 21:12:48.803879 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:48.803925 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:48.804449 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:48.804744 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:48.804747 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:48.805315 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:48.805522 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:48.805527 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:48.806225 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:48.806226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:48.807080 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:48.807082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:48.807488 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:48.807489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:48.807615 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:48.808155 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:48.808165 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:48.809552 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:48.811074 (+ 1522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:48.811080 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:48.811082 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:48.811309 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:48.811485 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:48.811486 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:48.811486 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:48.811586 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:48.811820 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:48.811826 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:48.812000 (+ 174us) client_negotiation.cc:769] Sending connection context
0924 21:12:48.812173 (+ 173us) client_negotiation.cc:240] Negotiation successful
0924 21:12:48.812351 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":81,"threads_started":1}
I20260924 21:12:48.813122 15347 heartbeater.cc:347] Connected to a master server at 127.6.210.190:37673
I20260924 21:12:48.813223 15347 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:48.813426 15347 heartbeater.cc:511] Master 127.6.210.190:37673 requested a full tablet report, sending...
I20260924 21:12:48.813931 15155 ts_manager.cc:194] Registered new tserver with Master: e9a809e392de45bdb29ec0f4bea7d2c7 (127.6.210.129:42923)
I20260924 21:12:48.814616 15155 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.129@KRBTEST.COM'} at 127.6.210.129:48473
WARNING: no policy specified for kudu/127.6.210.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
I20260924 21:12:48.837682 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.130:0
--local_ip_for_outbound_sockets=127.6.210.130
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:37673
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:48.918318 15352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:48.918520 15352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:48.918543 15352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:48.919637 15352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:48.919672 15352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:48.919698 15352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.130
I20260924 21:12:48.921147 15352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:37673
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.130
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:48.921413 15352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:48.921590 15352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:48.926771 15360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:48.926802 15357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:48.926772 15358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:48.927073 15352 server_base.cc:1034] running on GCE node
I20260924 21:12:48.927331 15352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:48.927570 15352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:48.928723 15352 hybrid_clock.cc:648] HybridClock initialized: now 1790284368928711 us; error 27 us; skew 500 ppm
Sep 24 21:12:48 dist-test-slave-q276 krb5kdc[15111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284368, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:48.931317 15352 init.cc:377] Logged in from keytab as kudu/127.6.210.130@KRBTEST.COM (short username kudu)
I20260924 21:12:48.931690 15352 webserver.cc:460] Webserver started at http://127.6.210.130:32813/ using document root <none> and password file <none>
I20260924 21:12:48.931872 15352 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:48.931912 15352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:48.932047 15352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:48.933034 15352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-1/data/instance:
uuid: "d5da6f8e619045aea3d39351238f19fb"
format_stamp: "Formatted at 2026-09-24 21:12:48 on dist-test-slave-q276"
server_key: "01fc6a989a3370a2d1f12b3a0096cc35"
server_key_iv: "d55ae6352c2f35dd6e17a6c87ded0626"
server_key_version: "encryptionkey@0"
I20260924 21:12:48.933321 15352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance:
uuid: "d5da6f8e619045aea3d39351238f19fb"
format_stamp: "Formatted at 2026-09-24 21:12:48 on dist-test-slave-q276"
server_key: "01fc6a989a3370a2d1f12b3a0096cc35"
server_key_iv: "d55ae6352c2f35dd6e17a6c87ded0626"
server_key_version: "encryptionkey@0"
I20260924 21:12:48.934635 15352 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:12:48.935307 15367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:48.935490 15352 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:12:48.935577 15352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-1/wal
uuid: "d5da6f8e619045aea3d39351238f19fb"
format_stamp: "Formatted at 2026-09-24 21:12:48 on dist-test-slave-q276"
server_key: "01fc6a989a3370a2d1f12b3a0096cc35"
server_key_iv: "d55ae6352c2f35dd6e17a6c87ded0626"
server_key_version: "encryptionkey@0"
I20260924 21:12:48.935649 15352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:48.943289 15352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:48.945106 15352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:48.945259 15352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:48.945466 15352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:48.945762 15352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:48.945822 15352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:48.945860 15352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:48.945887 15352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:48.951468 15352 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.130:45277
I20260924 21:12:48.951546 15480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.130:45277 every 8 connection(s)
I20260924 21:12:48.951858 15352 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
I20260924 21:12:48.952425 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 15352
I20260924 21:12:48.952545 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance
I20260924 21:12:48.952750 6986 external_mini_cluster.cc:1428] Setting key 2bd640b2b0195a88fbdb01102abce61f
Sep 24 21:12:48 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284368, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:48.961604 15196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:48.952590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:55551 (local address 127.6.210.190:37673)
0924 21:12:48.952636 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:48.952638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:48.952857 (+ 219us) server_negotiation.cc:407] Connection header received
0924 21:12:48.953437 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:48.953439 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:48.953451 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:48.953503 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:48.954522 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:48.955013 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:48.956030 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:48.956337 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:48.956409 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:48.958592 (+ 2183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:48.958594 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:48.958594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:48.958602 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:48.960198 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:48.960585 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:48.960587 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:48.960587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:48.960644 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:48.960885 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:48.960886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:48.960887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:48.961033 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:48.961114 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:48.961424 (+ 310us) server_negotiation.cc:299] Negotiation successful
0924 21:12:48.961506 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
WARNING: no policy specified for kudu/127.6.210.131@KRBTEST.COM; defaulting to no policy
I20260924 21:12:48.961755 15483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:48.952558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.130:55551)
0924 21:12:48.952719 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:48.952754 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:48.953323 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:48.953575 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:48.953579 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:48.954144 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:48.954423 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:48.954428 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:48.955126 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:48.955127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:48.955927 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:48.955929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:48.956422 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:48.956423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:48.956537 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:48.957071 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:48.957079 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:48.958502 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:48.960288 (+ 1786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:48.960293 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:48.960295 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:48.960506 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:48.960721 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:48.960722 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:48.960722 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:48.960817 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:48.961141 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:48.961147 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:48.961271 (+ 124us) client_negotiation.cc:769] Sending connection context
0924 21:12:48.961375 (+ 104us) client_negotiation.cc:240] Negotiation successful
0924 21:12:48.961513 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":89,"threads_started":1}
I20260924 21:12:48.962491 15481 heartbeater.cc:347] Connected to a master server at 127.6.210.190:37673
I20260924 21:12:48.962620 15481 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:48.962806 15481 heartbeater.cc:511] Master 127.6.210.190:37673 requested a full tablet report, sending...
I20260924 21:12:48.963330 15155 ts_manager.cc:194] Registered new tserver with Master: d5da6f8e619045aea3d39351238f19fb (127.6.210.130:45277)
I20260924 21:12:48.963881 15155 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.130@KRBTEST.COM'} at 127.6.210.130:55551
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
I20260924 21:12:48.979542 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.131:0
--local_ip_for_outbound_sockets=127.6.210.131
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:37673
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:49.062036 15486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:49.062318 15486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:49.062376 15486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:49.063584 15486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:49.063652 15486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:49.063697 15486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.131
I20260924 21:12:49.065119 15486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:37673
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.131
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:49.065385 15486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:49.065629 15486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:49.070762 15491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:49.070748 15492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:49.070729 15494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:49.070779 15486 server_base.cc:1034] running on GCE node
I20260924 21:12:49.071177 15486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:49.071424 15486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:49.072597 15486 hybrid_clock.cc:648] HybridClock initialized: now 1790284369072590 us; error 37 us; skew 500 ppm
Sep 24 21:12:49 dist-test-slave-q276 krb5kdc[15111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284369, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:49.075338 15486 init.cc:377] Logged in from keytab as kudu/127.6.210.131@KRBTEST.COM (short username kudu)
I20260924 21:12:49.075800 15486 webserver.cc:460] Webserver started at http://127.6.210.131:38395/ using document root <none> and password file <none>
I20260924 21:12:49.076031 15486 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:49.076099 15486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:49.076248 15486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:12:49.077342 15486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-2/data/instance:
uuid: "c5e6cca1dd0146e3a5f9e29b77957353"
format_stamp: "Formatted at 2026-09-24 21:12:49 on dist-test-slave-q276"
server_key: "d3054796282385c1d75b01e730469771"
server_key_iv: "e681ba35b4929aed03fbda7b2a52f3b3"
server_key_version: "encryptionkey@0"
I20260924 21:12:49.077718 15486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance:
uuid: "c5e6cca1dd0146e3a5f9e29b77957353"
format_stamp: "Formatted at 2026-09-24 21:12:49 on dist-test-slave-q276"
server_key: "d3054796282385c1d75b01e730469771"
server_key_iv: "e681ba35b4929aed03fbda7b2a52f3b3"
server_key_version: "encryptionkey@0"
I20260924 21:12:49.079173 15486 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:12:49.079949 15501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:49.080142 15486 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:49.080242 15486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-2/wal
uuid: "c5e6cca1dd0146e3a5f9e29b77957353"
format_stamp: "Formatted at 2026-09-24 21:12:49 on dist-test-slave-q276"
server_key: "d3054796282385c1d75b01e730469771"
server_key_iv: "e681ba35b4929aed03fbda7b2a52f3b3"
server_key_version: "encryptionkey@0"
I20260924 21:12:49.080332 15486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:49.093652 15486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:49.095788 15486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:49.095948 15486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:49.096177 15486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:49.096479 15486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:49.096540 15486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:49.096582 15486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:49.096609 15486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:49.102540 15486 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.131:40595
I20260924 21:12:49.102631 15614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.131:40595 every 8 connection(s)
I20260924 21:12:49.102983 15486 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
I20260924 21:12:49.104426 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 15486
I20260924 21:12:49.104563 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance
I20260924 21:12:49.104743 6986 external_mini_cluster.cc:1428] Setting key f92f6dbc0209afebfd712bcd1a6cbd5b
Sep 24 21:12:49 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284369, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:12:49.113233 15196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:49.103842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:32929 (local address 127.6.210.190:37673)
0924 21:12:49.103903 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:49.103905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:49.104151 (+ 246us) server_negotiation.cc:407] Connection header received
0924 21:12:49.104699 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:49.104700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:49.104712 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:49.104769 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:49.105872 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.106458 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.107447 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.107657 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.107748 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:49.110183 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:49.110185 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:49.110185 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:49.110193 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:49.111735 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:49.112239 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:49.112241 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:49.112242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:49.112304 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:49.112560 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:49.112561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:49.112562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:49.112684 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:49.112762 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:49.113045 (+ 283us) server_negotiation.cc:299] Negotiation successful
0924 21:12:49.113135 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 21:12:49.113440 15617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:49.103832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.131:32929)
0924 21:12:49.104006 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:49.104047 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:49.104563 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:49.104841 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:49.104845 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:49.105574 (+ 729us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:49.105775 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:49.105781 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.106547 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.106549 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:49.107353 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:49.107356 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.107758 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.107759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:49.107885 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:49.108669 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:49.108678 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:49.110082 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:12:49.111846 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:49.111853 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:49.111856 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:49.112140 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:49.112373 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:12:49.112374 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:12:49.112375 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:12:49.112484 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:12:49.112753 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:12:49.112759 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:12:49.112899 (+ 140us) client_negotiation.cc:769] Sending connection context
0924 21:12:49.113003 (+ 104us) client_negotiation.cc:240] Negotiation successful
0924 21:12:49.113187 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":99,"threads_started":1}
I20260924 21:12:49.113999 15615 heartbeater.cc:347] Connected to a master server at 127.6.210.190:37673
I20260924 21:12:49.114089 15615 heartbeater.cc:464] Registering TS with master...
I20260924 21:12:49.114326 15615 heartbeater.cc:511] Master 127.6.210.190:37673 requested a full tablet report, sending...
I20260924 21:12:49.114749 15155 ts_manager.cc:194] Registered new tserver with Master: c5e6cca1dd0146e3a5f9e29b77957353 (127.6.210.131:40595)
I20260924 21:12:49.115075 15155 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.131@KRBTEST.COM'} at 127.6.210.131:32929
I20260924 21:12:49.116453 6986 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.6.210.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.210.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.210.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.129.keytab.
Entry for principal oryx/127.6.210.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.129.keytab.
WARNING: no policy specified for oryx/127.6.210.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.210.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.210.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.130.keytab.
Entry for principal oryx/127.6.210.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.130.keytab.
WARNING: no policy specified for oryx/127.6.210.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.210.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.210.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.131.keytab.
Entry for principal oryx/127.6.210.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.131.keytab.
I20260924 21:12:49.189337 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 15218
I20260924 21:12:49.194028 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 15352
I20260924 21:12:49.198362 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 15486
I20260924 21:12:49.202584 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 15124
I20260924 21:12:49.207506 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:37673
--webserver_interface=127.6.210.190
--webserver_port=39251
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:37673
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:49.287575 15624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:49.287787 15624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:49.287810 15624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:49.288872 15624 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:12:49.288906 15624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:49.288920 15624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:49.288933 15624 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:12:49.288945 15624 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:12:49.290328 15624 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:37673
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:37673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=39251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:49.290517 15624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:49.290681 15624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:49.295495 15629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:49.295557 15624 server_base.cc:1034] running on GCE node
W20260924 21:12:49.295500 15632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:49.295504 15630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:49.295930 15624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:49.296231 15624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:49.297400 15624 hybrid_clock.cc:648] HybridClock initialized: now 1790284369297399 us; error 38 us; skew 500 ppm
Sep 24 21:12:49 dist-test-slave-q276 krb5kdc[15111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284369, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:49.299875 15624 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:12:49.300305 15624 webserver.cc:460] Webserver started at http://127.6.210.190:39251/ using document root <none> and password file <none>
I20260924 21:12:49.300536 15624 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:49.300619 15624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:49.302083 15624 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:49.302919 15639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:49.303107 15624 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 21:12:49.303215 15624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "236719135af64bbea3ef77417f3aa47b"
format_stamp: "Formatted at 2026-09-24 21:12:48 on dist-test-slave-q276"
server_key: "3c2b2296b1bfe776fbb01a75adf4922b"
server_key_iv: "ab0d3d5b13b3ac1c9dccd55e78bbe46a"
server_key_version: "encryptionkey@0"
I20260924 21:12:49.303633 15624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:49.313941 15624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:49.315826 15624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:49.315938 15624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:49.319809 15624 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:37673
I20260924 21:12:49.319844 15691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:37673 every 8 connection(s)
I20260924 21:12:49.320231 15624 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:12:49.321671 15692 sys_catalog.cc:263] Verifying existing consensus state
I20260924 21:12:49.321954 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 15624
I20260924 21:12:49.322288 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.129:42923
--local_ip_for_outbound_sockets=127.6.210.129
--tserver_master_addrs=127.6.210.190:37673
--webserver_port=40189
--webserver_interface=127.6.210.129
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.210.129
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.129.keytab with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:12:49.322642 15692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b: Bootstrap starting.
I20260924 21:12:49.330793 15692 log.cc:824] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b: Log is configured to *not* fsync() on all Append() calls
I20260924 21:12:49.332438 15692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 21:12:49.332721 15692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b: Bootstrap complete.
I20260924 21:12:49.335014 15692 raft_consensus.cc:348] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "236719135af64bbea3ef77417f3aa47b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37673 } }
I20260924 21:12:49.335263 15692 raft_consensus.cc:729] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 236719135af64bbea3ef77417f3aa47b, State: Initialized, Role: FOLLOWER
I20260924 21:12:49.335453 15692 consensus_queue.cc:260] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "236719135af64bbea3ef77417f3aa47b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37673 } }
I20260924 21:12:49.335548 15692 raft_consensus.cc:388] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:12:49.335603 15692 raft_consensus.cc:482] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:12:49.335649 15692 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [term 1 FOLLOWER]: Advancing to term 2
I20260924 21:12:49.336624 15692 raft_consensus.cc:504] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "236719135af64bbea3ef77417f3aa47b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37673 } }
I20260924 21:12:49.336710 15692 leader_election.cc:302] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 236719135af64bbea3ef77417f3aa47b; no voters:
I20260924 21:12:49.336953 15692 leader_election.cc:288] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 21:12:49.336987 15696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [term 2 FOLLOWER]: Leader election won for term 2
I20260924 21:12:49.337100 15696 raft_consensus.cc:686] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [term 2 LEADER]: Becoming Leader. State: Replica: 236719135af64bbea3ef77417f3aa47b, State: Running, Role: LEADER
I20260924 21:12:49.337188 15696 consensus_queue.cc:237] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "236719135af64bbea3ef77417f3aa47b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37673 } }
I20260924 21:12:49.337231 15692 sys_catalog.cc:564] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:12:49.337459 15698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 236719135af64bbea3ef77417f3aa47b. Latest consensus state: current_term: 2 leader_uuid: "236719135af64bbea3ef77417f3aa47b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "236719135af64bbea3ef77417f3aa47b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37673 } } }
I20260924 21:12:49.337658 15698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:49.337603 15697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "236719135af64bbea3ef77417f3aa47b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "236719135af64bbea3ef77417f3aa47b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37673 } } }
I20260924 21:12:49.337935 15697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b [sys.catalog]: This master's current role is: LEADER
I20260924 21:12:49.338004 15704 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:12:49.338551 15704 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:12:49.338707 15704 catalog_manager.cc:1259] Loaded cluster ID: 4a368ced3174462995ca0025b7222850
I20260924 21:12:49.338923 15704 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:12:49.339743 15704 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:12:49.339926 15704 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 236719135af64bbea3ef77417f3aa47b: Loaded TSK: 0
I20260924 21:12:49.340139 15704 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 21:12:49.420235 15694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:49.420509 15694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:49.420555 15694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:49.421937 15694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:49.422024 15694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:49.422084 15694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.129
I20260924 21:12:49.423885 15694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.129.keytab
--principal=oryx/127.6.210.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.129:42923
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.210.129
--webserver_port=40189
--tserver_master_addrs=127.6.210.190:37673
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.129
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:49.424211 15694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:49.424479 15694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:49.431447 15718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:49.431465 15719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:49.431675 15721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:49.433331 15694 server_base.cc:1034] running on GCE node
I20260924 21:12:49.433578 15694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:49.433884 15694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:49.435061 15694 hybrid_clock.cc:648] HybridClock initialized: now 1790284369435029 us; error 48 us; skew 500 ppm
Sep 24 21:12:49 dist-test-slave-q276 krb5kdc[15111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284369, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:49.438406 15694 init.cc:377] Logged in from keytab as oryx/127.6.210.129@KRBTEST.COM (short username oryx)
I20260924 21:12:49.438941 15694 webserver.cc:460] Webserver started at http://127.6.210.129:40189/ using document root <none> and password file <none>
I20260924 21:12:49.439230 15694 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:49.439308 15694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:49.441102 15694 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:12:49.441856 15728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:49.442088 15694 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:49.442184 15694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-0/wal
uuid: "e9a809e392de45bdb29ec0f4bea7d2c7"
format_stamp: "Formatted at 2026-09-24 21:12:48 on dist-test-slave-q276"
server_key: "bba249730873c5d44ab52216545b9072"
server_key_iv: "a3df7496397b7610290ca1cd54cd9996"
server_key_version: "encryptionkey@0"
I20260924 21:12:49.442711 15694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:49.458666 15694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:49.461407 15694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:49.461578 15694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:49.461850 15694 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:49.462212 15694 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:49.462301 15694 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:49.462352 15694 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:49.462371 15694 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:49.469681 15694 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.129:42923
I20260924 21:12:49.469715 15841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.129:42923 every 8 connection(s)
I20260924 21:12:49.470144 15694 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
Sep 24 21:12:49 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:49.477945 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 15694
Sep 24 21:12:49 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:49.478255 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.130:45277
--local_ip_for_outbound_sockets=127.6.210.130
--tserver_master_addrs=127.6.210.190:37673
--webserver_port=32813
--webserver_interface=127.6.210.130
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.210.130
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.130.keytab with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:12:49.479139 15844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:49.470938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.129:47927)
0924 21:12:49.471122 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:49.471173 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:49.471681 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:49.472155 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:49.472159 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:49.472796 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:49.473001 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:49.473006 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.474143 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.474144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:49.475003 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:49.475005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.475630 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.475631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:49.475764 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:49.476343 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:49.476352 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:49.478940 (+ 2588us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":86,"threads_started":1}
W20260924 21:12:49.479518 15842 heartbeater.cc:644] Failed to heartbeat to 127.6.210.190:37673 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.210.190:37673: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
W20260924 21:12:49.479519 15845 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:49.470995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:47927 (local address 127.6.210.190:37673)
0924 21:12:49.471213 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:49.471222 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:49.471291 (+ 69us) server_negotiation.cc:407] Connection header received
0924 21:12:49.471917 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:49.471923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:49.471967 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:49.472115 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:49.473138 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.474021 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.475097 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.475522 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.475654 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:49.479107 (+ 3453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.129:47927: BlockingRecv error: recv got EOF from 127.6.210.129:47927 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":104,"threads_started":1}
Sep 24 21:12:49 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:49 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:49.485385 15844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:49.479725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.129:33905)
0924 21:12:49.479826 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:49.479836 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:49.479873 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:49.480090 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:49.480092 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:49.480284 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:49.480344 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:49.480346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.481289 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.481290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:49.482080 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:49.482083 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.482626 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.482627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:49.482696 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:49.483216 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:49.483221 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:49.485313 (+ 2092us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":86}
W20260924 21:12:49.485608 15845 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:49.479787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:33905 (local address 127.6.210.190:37673)
0924 21:12:49.479941 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:49.479943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:49.479952 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:12:49.479979 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:49.479980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:49.479995 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:49.480066 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:49.480431 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.481185 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.482174 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.482541 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.482620 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:49.485428 (+ 2808us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.129:33905: BlockingRecv error: recv got EOF from 127.6.210.129:33905 (error 108)
Metrics: {"server-negotiator.queue_time_us":134}
Sep 24 21:12:49 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:49 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:49.491036 15844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:49.485693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.129:37039)
0924 21:12:49.485830 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:49.485840 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:49.485873 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:49.486157 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:49.486158 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:49.486318 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:49.486352 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:49.486353 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.487289 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.487291 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:49.487984 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:49.487987 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.488484 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.488485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:49.488534 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:49.489027 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:49.489032 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:49.490963 (+ 1931us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":95,"threads_started":1}
W20260924 21:12:49.491117 15845 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:49.485810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:37039 (local address 127.6.210.190:37673)
0924 21:12:49.486033 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:49.486035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:49.486045 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:12:49.486067 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:49.486068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:49.486080 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:49.486146 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:49.486417 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.487196 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.488072 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.488397 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.488477 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:49.491042 (+ 2565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.129:37039: BlockingRecv error: recv got EOF from 127.6.210.129:37039 (error 108)
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":91,"threads_started":1}
W20260924 21:12:49.491303 15842 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 21:12:49.561875 15846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:49.562084 15846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:49.562107 15846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:49.563375 15846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:49.563421 15846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:49.563447 15846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.130
I20260924 21:12:49.564886 15846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.130.keytab
--principal=oryx/127.6.210.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.130:45277
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.210.130
--webserver_port=32813
--tserver_master_addrs=127.6.210.190:37673
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.130
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:49.565125 15846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:49.565358 15846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:49.570530 15853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:49.570564 15856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:49.570602 15854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:49.570714 15846 server_base.cc:1034] running on GCE node
I20260924 21:12:49.570992 15846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:49.571210 15846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:49.572376 15846 hybrid_clock.cc:648] HybridClock initialized: now 1790284369572360 us; error 25 us; skew 500 ppm
Sep 24 21:12:49 dist-test-slave-q276 krb5kdc[15111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284369, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:49.574923 15846 init.cc:377] Logged in from keytab as oryx/127.6.210.130@KRBTEST.COM (short username oryx)
I20260924 21:12:49.575364 15846 webserver.cc:460] Webserver started at http://127.6.210.130:32813/ using document root <none> and password file <none>
I20260924 21:12:49.575608 15846 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:49.575698 15846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:49.577061 15846 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:49.577620 15863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:49.577790 15846 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:12:49.577888 15846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-1/wal
uuid: "d5da6f8e619045aea3d39351238f19fb"
format_stamp: "Formatted at 2026-09-24 21:12:48 on dist-test-slave-q276"
server_key: "01fc6a989a3370a2d1f12b3a0096cc35"
server_key_iv: "d55ae6352c2f35dd6e17a6c87ded0626"
server_key_version: "encryptionkey@0"
I20260924 21:12:49.578233 15846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:49.597101 15846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:49.599522 15846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:49.599696 15846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:49.599946 15846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:49.600324 15846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:49.600451 15846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:49.600538 15846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:49.600579 15846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:49.606701 15846 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.130:45277
I20260924 21:12:49.606762 15976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.130:45277 every 8 connection(s)
I20260924 21:12:49.607131 15846 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
Sep 24 21:12:49 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:49 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:49.613849 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 15846
I20260924 21:12:49.614166 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.131:40595
--local_ip_for_outbound_sockets=127.6.210.131
--tserver_master_addrs=127.6.210.190:37673
--webserver_port=38395
--webserver_interface=127.6.210.131
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:42351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.210.131
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.131.keytab with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:12:49.614619 15845 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:49.607880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:38829 (local address 127.6.210.190:37673)
0924 21:12:49.607925 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:49.607926 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:49.608129 (+ 203us) server_negotiation.cc:407] Connection header received
0924 21:12:49.608608 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:49.608609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:49.608622 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:49.608674 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:49.609595 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.610075 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.611079 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.611263 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.611328 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:49.614573 (+ 3245us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.130:38829: BlockingRecv error: recv got EOF from 127.6.210.130:38829 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 21:12:49.614660 15979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:49.607884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.130:38829)
0924 21:12:49.608027 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:49.608062 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:49.608523 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:49.608730 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:49.608734 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:49.609301 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:49.609510 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:49.609515 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.610159 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.610159 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:49.610981 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:49.610982 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.611398 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.611399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:49.611510 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:49.611964 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:49.611972 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:49.614426 (+ 2454us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":79,"threads_started":1}
W20260924 21:12:49.614954 15977 heartbeater.cc:644] Failed to heartbeat to 127.6.210.190:37673 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.210.190:37673: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Sep 24 21:12:49 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:49 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:49.619450 15979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:49.615213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.130:58995)
0924 21:12:49.615262 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:49.615275 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:49.615327 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:49.615454 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:49.615455 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:49.615584 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:49.615624 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:49.615626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.616248 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.616249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:49.616748 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:49.616760 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.617127 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.617128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:49.617193 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:49.617649 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:49.617654 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:49.619364 (+ 1710us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260924 21:12:49.619495 15845 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:49.615227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:58995 (local address 127.6.210.190:37673)
0924 21:12:49.615268 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:49.615270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:49.615311 (+ 41us) server_negotiation.cc:407] Connection header received
0924 21:12:49.615386 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:49.615386 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:49.615396 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:49.615450 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:49.615698 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.616161 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.616841 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.617042 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.617125 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:49.619433 (+ 2308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.130:58995: BlockingRecv error: recv got EOF from 127.6.210.130:58995 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Sep 24 21:12:49 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:49 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:49.624300 15979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:49.619945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.130:49655)
0924 21:12:49.620001 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:49.620013 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:49.620057 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:49.620186 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:49.620187 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:49.620322 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:49.620350 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:49.620351 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.620953 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.620954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:49.621447 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:49.621449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.621808 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.621809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:49.621862 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:49.622313 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:49.622318 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:49.624207 (+ 1889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":40}
W20260924 21:12:49.624332 15845 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:49.620029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:49655 (local address 127.6.210.190:37673)
0924 21:12:49.620067 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:49.620069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:49.620080 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:12:49.620104 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:49.620104 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:49.620114 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:49.620183 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:49.620432 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.620877 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.621543 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.621724 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.621806 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:49.624276 (+ 2470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.130:49655: BlockingRecv error: recv got EOF from 127.6.210.130:49655 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260924 21:12:49.624661 15977 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 21:12:49.696439 15980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:12:49.696653 15980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:12:49.696678 15980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:12:49.697916 15980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:12:49.697965 15980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:12:49.698029 15980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.131
I20260924 21:12:49.699651 15980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:42351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/krb5kdc/oryx_127.6.210.131.keytab
--principal=oryx/127.6.210.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.131:40595
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.210.131
--webserver_port=38395
--tserver_master_addrs=127.6.210.190:37673
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.131
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:12:49.699913 15980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:12:49.700155 15980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:12:49.705259 15986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:12:49.705271 15988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:49.705302 15980 server_base.cc:1034] running on GCE node
W20260924 21:12:49.705307 15985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:12:49.705790 15980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:12:49.705999 15980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:12:49.707134 15980 hybrid_clock.cc:648] HybridClock initialized: now 1790284369707123 us; error 30 us; skew 500 ppm
Sep 24 21:12:49 dist-test-slave-q276 krb5kdc[15111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284369, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.210.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:12:49.709693 15980 init.cc:377] Logged in from keytab as oryx/127.6.210.131@KRBTEST.COM (short username oryx)
I20260924 21:12:49.710134 15980 webserver.cc:460] Webserver started at http://127.6.210.131:38395/ using document root <none> and password file <none>
I20260924 21:12:49.710417 15980 fs_manager.cc:359] Metadata directory not provided
I20260924 21:12:49.710493 15980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:12:49.711804 15980 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:12:49.712394 15995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:49.712561 15980 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:12:49.712639 15980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-2/wal
uuid: "c5e6cca1dd0146e3a5f9e29b77957353"
format_stamp: "Formatted at 2026-09-24 21:12:49 on dist-test-slave-q276"
server_key: "d3054796282385c1d75b01e730469771"
server_key_iv: "e681ba35b4929aed03fbda7b2a52f3b3"
server_key_version: "encryptionkey@0"
I20260924 21:12:49.712954 15980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:12:49.729411 15980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:12:49.731411 15980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:12:49.731526 15980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:12:49.731711 15980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:12:49.732038 15980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:12:49.732074 15980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:49.732096 15980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:12:49.732148 15980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:12:49.738032 15980 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.131:40595
I20260924 21:12:49.738075 16108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.131:40595 every 8 connection(s)
I20260924 21:12:49.738484 15980 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
I20260924 21:12:49.739334 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 15980
Sep 24 21:12:49 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:49 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
W20260924 21:12:49.750850 15845 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:49.739265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:56111 (local address 127.6.210.190:37673)
0924 21:12:49.739311 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:49.739314 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:49.739612 (+ 298us) server_negotiation.cc:407] Connection header received
0924 21:12:49.740304 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:49.740304 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:49.740310 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:49.740340 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:49.741427 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.741893 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.742971 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.743170 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.743257 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:49.750804 (+ 7547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.131:56111: BlockingRecv error: recv got EOF from 127.6.210.131:56111 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 21:12:49.751015 16111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:49.739288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.131:56111)
0924 21:12:49.739469 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:49.739517 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:49.740202 (+ 685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:49.740436 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:49.740440 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:49.741137 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:49.741330 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:49.741335 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.741987 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.741988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:49.742869 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:49.742871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.743268 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.743269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:49.743393 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:49.748121 (+ 4728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:49.748129 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:49.750726 (+ 2597us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":94,"threads_started":1}
W20260924 21:12:49.751276 16109 heartbeater.cc:644] Failed to heartbeat to 127.6.210.190:37673 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.210.190:37673: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
I20260924 21:12:49.752400 15848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:49.739843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52286 (local address 127.6.210.190:37673)
0924 21:12:49.739890 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:49.739891 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:49.739938 (+ 47us) server_negotiation.cc:407] Connection header received
0924 21:12:49.740021 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:49.740022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:49.740032 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:49.740091 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:49.740871 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.741358 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.742131 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.742409 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.742503 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:49.748832 (+ 6329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:12:49.748836 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:12:49.748836 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:12:49.748846 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:12:49.750887 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:49.751294 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:49.751298 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:49.751298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:49.751421 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:12:49.751653 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:12:49.751654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:12:49.751655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:12:49.751937 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:12:49.752085 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0924 21:12:49.752198 (+ 113us) server_negotiation.cc:299] Negotiation successful
0924 21:12:49.752312 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Sep 24 21:12:49 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:49 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:49.756078 16111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:49.751519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.131:39087)
0924 21:12:49.751565 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:49.751576 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:49.751617 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:49.751764 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:49.751765 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:49.751870 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:49.751914 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:49.751916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.752777 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.752779 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:49.753516 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:49.753518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.753860 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.753861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:49.753908 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:49.754340 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:49.754345 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:49.756020 (+ 1675us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":26}
W20260924 21:12:49.756137 15845 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:49.751563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:39087 (local address 127.6.210.190:37673)
0924 21:12:49.751599 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:49.751601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:49.751609 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:12:49.751679 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:49.751679 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:49.751690 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:49.751749 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:49.751982 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.752718 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.753605 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.753801 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.753857 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:49.756099 (+ 2242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.131:39087: BlockingRecv error: recv got EOF from 127.6.210.131:39087 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
Sep 24 21:12:49 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:49 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:49.760272 16111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:49.756371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.131:53671)
0924 21:12:49.756420 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:49.756430 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:49.756487 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:49.756618 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:49.756620 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:49.756737 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:49.756774 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:49.756775 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.757392 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.757393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:49.757872 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:49.757873 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.758227 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.758228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:49.758308 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:49.758657 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:49.758661 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:49.760188 (+ 1527us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260924 21:12:49.760303 15845 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:49.756429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:53671 (local address 127.6.210.190:37673)
0924 21:12:49.756482 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:49.756484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:49.756492 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:12:49.756547 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:49.756547 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:49.756557 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:49.756608 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:49.756847 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.757304 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.757964 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:49.758147 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:49.758227 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:49.760251 (+ 2024us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.131:53671: BlockingRecv error: recv got EOF from 127.6.210.131:53671 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260924 21:12:49.760449 16109 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 24 21:12:50 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:50 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:50.496608 16113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:50.491711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.129:45751)
0924 21:12:50.491848 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:50.491859 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:50.491917 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:50.492082 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:50.492083 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:50.492260 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:50.492311 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:50.492313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:50.493063 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:50.493064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:50.493661 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:50.493663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:50.494003 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:50.494004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:50.494045 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:50.494509 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:50.494514 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:50.496529 (+ 2015us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
W20260924 21:12:50.496738 16114 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:50.491821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:45751 (local address 127.6.210.190:37673)
0924 21:12:50.491956 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:50.491958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:50.491967 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:12:50.491990 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:50.491990 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:50.492000 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:50.492061 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:50.492387 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:50.492980 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:50.493758 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:50.493941 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:50.493988 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:50.496631 (+ 2643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.129:45751: BlockingRecv error: recv got EOF from 127.6.210.129:45751 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
Sep 24 21:12:50 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:50 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:50.629858 16115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:50.625184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.130:47497)
0924 21:12:50.625334 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:50.625350 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:50.625413 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:50.625571 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:50.625572 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:50.625744 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:50.625794 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:50.625796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:50.626462 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:50.626463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:50.627091 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:50.627093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:50.627484 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:50.627485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:50.627534 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:50.627953 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:50.627957 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:50.629788 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
W20260924 21:12:50.629930 16114 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:50.625270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:47497 (local address 127.6.210.190:37673)
0924 21:12:50.625357 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:50.625359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:50.625398 (+ 39us) server_negotiation.cc:407] Connection header received
0924 21:12:50.625476 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:50.625476 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:50.625487 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:50.625544 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:50.625862 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:50.626390 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:50.627174 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:50.627387 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:50.627447 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:50.629879 (+ 2432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.130:47497: BlockingRecv error: recv got EOF from 127.6.210.130:47497 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 24 21:12:50 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:50 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:50.765879 16116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:50.761050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.131:37269)
0924 21:12:50.761232 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:50.761242 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:50.761304 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:50.761477 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:50.761478 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:50.761657 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:50.761711 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:50.761713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:50.762381 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:50.762382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:50.763139 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:50.763142 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:50.763526 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:50.763527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:50.763575 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:50.764039 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:50.764044 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:50.765813 (+ 1769us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":92,"threads_started":1}
W20260924 21:12:50.766044 16114 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:50.761135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:37269 (local address 127.6.210.190:37673)
0924 21:12:50.761209 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:50.761211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:50.761343 (+ 132us) server_negotiation.cc:407] Connection header received
0924 21:12:50.761370 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:50.761371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:50.761383 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:50.761428 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:50.761787 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:50.762314 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:50.763220 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:50.763443 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:50.763499 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:50.765904 (+ 2405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.131:37269: BlockingRecv error: recv got EOF from 127.6.210.131:37269 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 24 21:12:51 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:51 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:51.501421 16117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:51.497167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.129:47147)
0924 21:12:51.497320 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:51.497330 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:51.497380 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:51.497521 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:51.497522 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:51.497676 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:51.497723 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:51.497725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:51.498333 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:51.498335 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:51.498849 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:51.498850 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:51.499153 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:51.499154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:51.499201 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:51.499633 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:51.499638 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:51.501364 (+ 1726us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":82,"threads_started":1}
W20260924 21:12:51.501567 16118 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:51.497259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:47147 (local address 127.6.210.190:37673)
0924 21:12:51.497395 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:51.497397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:51.497413 (+ 16us) server_negotiation.cc:407] Connection header received
0924 21:12:51.497436 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:51.497437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:51.497447 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:51.497517 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:51.497782 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:51.498278 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:51.498910 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:51.499090 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:51.499139 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:51.501476 (+ 2337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.129:47147: BlockingRecv error: recv got EOF from 127.6.210.129:47147 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
Sep 24 21:12:51 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:51 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:51.635394 16119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:51.630447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.130:52637)
0924 21:12:51.630751 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:51.630761 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:51.630850 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:51.631011 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:51.631012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:51.631193 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:51.631242 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:51.631244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:51.631915 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:51.631916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:51.632475 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:51.632477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:51.632912 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:51.632913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:51.632972 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:51.633500 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:51.633506 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:51.635300 (+ 1794us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":97,"threads_started":1}
W20260924 21:12:51.635454 16118 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:51.630534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:52637 (local address 127.6.210.190:37673)
0924 21:12:51.630716 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:51.630719 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:51.630828 (+ 109us) server_negotiation.cc:407] Connection header received
0924 21:12:51.630934 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:51.630935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:51.630948 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:51.630999 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:51.631327 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:51.631822 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:51.632573 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:51.632830 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:51.632896 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:51.635392 (+ 2496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.130:52637: BlockingRecv error: recv got EOF from 127.6.210.130:52637 (error 108)
Metrics: {"server-negotiator.queue_time_us":148}
Sep 24 21:12:51 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:51 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:51.770974 16120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:51.766325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.131:56797)
0924 21:12:51.766495 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:51.766507 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:51.766560 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:51.766669 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:51.766670 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:51.766862 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:51.766910 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:51.766911 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:51.767535 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:51.767535 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:51.768056 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:51.768058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:51.768399 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:51.768400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:51.768448 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:51.768903 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:51.768938 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:51.770912 (+ 1974us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":71,"threads_started":1}
W20260924 21:12:51.771083 16118 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:51.766473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:56797 (local address 127.6.210.190:37673)
0924 21:12:51.766537 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:51.766539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:51.766551 (+ 12us) server_negotiation.cc:407] Connection header received
0924 21:12:51.766606 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:51.766606 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:51.766617 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:51.766664 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:51.766971 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:51.767463 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:51.768127 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:51.768316 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:51.768368 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:51.771014 (+ 2646us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.131:56797: BlockingRecv error: recv got EOF from 127.6.210.131:56797 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 24 21:12:52 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:52 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:52.508389 16121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:52.501935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.129:44897)
0924 21:12:52.502092 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:52.502108 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:52.502180 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:52.502551 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:52.502552 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:52.502800 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:52.502866 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:52.502869 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:52.503944 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:52.503945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:52.504743 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:52.504746 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:52.505199 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:52.505200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:52.505267 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:52.505879 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:52.505884 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:52.508283 (+ 2399us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":104,"threads_started":1}
W20260924 21:12:52.508458 16122 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:52.502009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:44897 (local address 127.6.210.190:37673)
0924 21:12:52.502145 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:52.502147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:52.502159 (+ 12us) server_negotiation.cc:407] Connection header received
0924 21:12:52.502359 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:52.502360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:52.502377 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:52.502476 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:52.502972 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:52.503847 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:52.504855 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:52.505095 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:52.505190 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:52.508376 (+ 3186us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.129:44897: BlockingRecv error: recv got EOF from 127.6.210.129:44897 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":68,"threads_started":1}
Sep 24 21:12:52 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:52 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:52.641083 16123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:52.635941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.130:39257)
0924 21:12:52.636136 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:52.636150 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:52.636258 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:52.636409 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:52.636410 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:52.636680 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:52.636728 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:52.636730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:52.637495 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:52.637496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:52.638134 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:52.638136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:52.638611 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:52.638612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:52.638688 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:52.639240 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:52.639245 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:52.641024 (+ 1779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":97,"threads_started":1}
W20260924 21:12:52.641266 16122 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:52.636059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:39257 (local address 127.6.210.190:37673)
0924 21:12:52.636140 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:52.636143 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:52.636236 (+ 93us) server_negotiation.cc:407] Connection header received
0924 21:12:52.636338 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:52.636339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:52.636351 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:52.636405 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:52.636833 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:52.637371 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:52.638233 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:52.638501 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:52.638591 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:52.641197 (+ 2606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.130:39257: BlockingRecv error: recv got EOF from 127.6.210.130:39257 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 24 21:12:52 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:52 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:52.775956 16124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:52.771445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.131:38803)
0924 21:12:52.771618 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:52.771630 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:52.771693 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:52.771828 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:52.771828 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:52.771983 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:52.772031 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:52.772032 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:52.772636 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:52.772636 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:52.773145 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:52.773146 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:52.773458 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:52.773458 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:52.773499 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:52.774021 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:52.774027 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:52.775858 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":103,"threads_started":1}
W20260924 21:12:52.776001 16122 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:52.771528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:38803 (local address 127.6.210.190:37673)
0924 21:12:52.771590 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:52.771593 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:52.771698 (+ 105us) server_negotiation.cc:407] Connection header received
0924 21:12:52.771750 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:52.771751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:52.771764 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:52.771804 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:52.772094 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:52.772571 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:52.773217 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:52.773392 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:52.773442 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:52.775938 (+ 2496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.131:38803: BlockingRecv error: recv got EOF from 127.6.210.131:38803 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 24 21:12:53 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:53 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:53.513959 16125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:53.509011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.129:49035)
0924 21:12:53.509164 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:53.509174 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:53.509227 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:53.509468 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:53.509469 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:53.509630 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:53.509677 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:53.509678 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:53.510534 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:53.510535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:53.511077 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:53.511079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:53.511402 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:53.511402 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:53.511439 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:53.511873 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:53.511878 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:53.513902 (+ 2024us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":69,"threads_started":1}
W20260924 21:12:53.514073 16126 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:53.509087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:49035 (local address 127.6.210.190:37673)
0924 21:12:53.509309 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:53.509311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:53.509320 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:12:53.509344 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:53.509345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:53.509356 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:53.509445 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:53.509779 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:53.510437 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:53.511152 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:53.511337 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:53.511391 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:53.513995 (+ 2604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.129:49035: BlockingRecv error: recv got EOF from 127.6.210.129:49035 (error 108)
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":155,"threads_started":1}
Sep 24 21:12:53 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:53 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:53.646132 16127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:53.641536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.130:38393)
0924 21:12:53.641672 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:53.641683 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:53.641745 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:53.641873 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:53.641874 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:53.642031 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:53.642077 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:53.642078 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:53.642808 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:53.642809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:53.643340 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:53.643341 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:53.643696 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:53.643697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:53.643734 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:53.644194 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:53.644227 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:53.646018 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":71,"threads_started":1}
W20260924 21:12:53.646196 16126 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:53.641614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:38393 (local address 127.6.210.190:37673)
0924 21:12:53.641699 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:53.641701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:53.641744 (+ 43us) server_negotiation.cc:407] Connection header received
0924 21:12:53.641792 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:53.641793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:53.641804 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:53.641858 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:53.642137 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:53.642733 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:53.643434 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:53.643626 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:53.643691 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:53.646150 (+ 2459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.130:38393: BlockingRecv error: recv got EOF from 127.6.210.130:38393 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 24 21:12:53 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:53 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:53.781215 16128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:53.776470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.131:54015)
0924 21:12:53.776614 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:53.776627 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:53.776708 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:53.776868 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:53.776869 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:53.777034 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:53.777082 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:53.777083 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:53.777928 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:53.777929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:53.778522 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:53.778523 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:53.778831 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:53.778831 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:53.778870 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:53.779363 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:53.779368 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:53.781157 (+ 1789us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":53,"threads_started":1}
W20260924 21:12:53.781356 16126 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:53.776588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:54015 (local address 127.6.210.190:37673)
0924 21:12:53.776656 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:53.776658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:53.776690 (+ 32us) server_negotiation.cc:407] Connection header received
0924 21:12:53.776764 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:53.776765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:53.776789 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:53.776839 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:53.777141 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:53.777846 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:53.778585 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:53.778764 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:53.778818 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:53.781267 (+ 2449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.131:54015: BlockingRecv error: recv got EOF from 127.6.210.131:54015 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 24 21:12:54 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:54 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:54.519284 16129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:54.514533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.129:47583)
0924 21:12:54.514681 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:54.514691 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:54.514756 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:54.514944 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:54.514945 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:54.515149 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:54.515211 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:54.515213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:54.515915 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:54.515915 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:54.516510 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:54.516511 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:54.516860 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:54.516861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:54.516898 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:54.517407 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:54.517413 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:54.519220 (+ 1807us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":84,"threads_started":1}
W20260924 21:12:54.519449 16130 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:54.514583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:47583 (local address 127.6.210.190:37673)
0924 21:12:54.514739 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:54.514741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:54.514754 (+ 13us) server_negotiation.cc:407] Connection header received
0924 21:12:54.514837 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:54.514838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:54.514852 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:54.514946 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:54.515313 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:54.515851 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:54.516581 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:54.516774 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:54.516856 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:54.519342 (+ 2486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.129:47583: BlockingRecv error: recv got EOF from 127.6.210.129:47583 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":74,"threads_started":1}
Sep 24 21:12:54 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:54 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:54.651351 16131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:54.646643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.130:51299)
0924 21:12:54.646841 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:54.646855 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:54.646924 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:54.647077 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:54.647078 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:54.647268 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:54.647317 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:54.647320 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:54.648065 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:54.648066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:54.648602 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:54.648604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:54.648973 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:54.648974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:54.649029 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:54.649474 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:54.649478 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:54.651255 (+ 1777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":67,"threads_started":1}
W20260924 21:12:54.651407 16130 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:54.646787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:51299 (local address 127.6.210.190:37673)
0924 21:12:54.646845 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:54.646848 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:54.646903 (+ 55us) server_negotiation.cc:407] Connection header received
0924 21:12:54.646987 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:54.646988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:54.647006 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:54.647071 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:54.647409 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:54.647986 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:54.648688 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:54.648888 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:54.648973 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:54.651342 (+ 2369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.130:51299: BlockingRecv error: recv got EOF from 127.6.210.130:51299 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 24 21:12:54 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:54 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:54.791462 16132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:54.781727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.131:52055)
0924 21:12:54.781914 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:54.781934 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:54.781997 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:54.782126 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:54.782127 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:54.782309 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:54.782354 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:54.782356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:54.782962 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:54.782963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:54.783496 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:54.783498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:54.783806 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:54.783807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:54.783845 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:54.789525 (+ 5680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:54.789530 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:54.791404 (+ 1874us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":113,"threads_started":1}
W20260924 21:12:54.791574 16130 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:54.781774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:52055 (local address 127.6.210.190:37673)
0924 21:12:54.781839 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:54.781842 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:54.781995 (+ 153us) server_negotiation.cc:407] Connection header received
0924 21:12:54.782045 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:54.782045 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:54.782059 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:54.782111 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:54.782419 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:54.782898 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:54.783570 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:54.783744 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:54.783795 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:54.791498 (+ 7703us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.131:52055: BlockingRecv error: recv got EOF from 127.6.210.131:52055 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 24 21:12:55 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:55 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:55.525228 16133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:55.519831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.129:55199)
0924 21:12:55.519999 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:55.520014 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:55.520108 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:55.520435 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:55.520436 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:55.520616 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:55.520663 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:55.520665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:55.521386 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:55.521387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:55.521906 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:55.521907 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:55.522258 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:55.522259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:55.522374 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:55.522830 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:55.522835 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:55.525160 (+ 2325us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":69,"threads_started":1}
W20260924 21:12:55.525339 16134 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:55.519981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:55199 (local address 127.6.210.190:37673)
0924 21:12:55.520220 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:55.520222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:55.520231 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:12:55.520284 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:55.520285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:55.520296 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:55.520372 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:55.520763 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:55.521295 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:55.521984 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:55.522188 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:55.522244 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:55.525258 (+ 3014us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.129:55199: BlockingRecv error: recv got EOF from 127.6.210.129:55199 (error 108)
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":120,"threads_started":1}
Sep 24 21:12:55 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:55 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:55.656804 16135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:55.651874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.130:53309)
0924 21:12:55.652056 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:55.652067 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:55.652135 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:55.652291 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:55.652292 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:55.652455 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:55.652503 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:55.652504 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:55.653146 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:55.653147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:55.653717 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:55.653719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:55.654077 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:55.654078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:55.654128 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:55.654734 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:55.654740 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:55.656652 (+ 1912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":104,"threads_started":1}
W20260924 21:12:55.656827 16134 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:55.651946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:53309 (local address 127.6.210.190:37673)
0924 21:12:55.652032 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:55.652034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:55.652137 (+ 103us) server_negotiation.cc:407] Connection header received
0924 21:12:55.652202 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:55.652203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:55.652214 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:55.652273 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:55.652570 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:55.653070 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:55.653804 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:55.653990 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:55.654047 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:55.656747 (+ 2700us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.130:53309: BlockingRecv error: recv got EOF from 127.6.210.130:53309 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 24 21:12:55 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:55 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:55.796887 16136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:55.792003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.131:38761)
0924 21:12:55.792177 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:55.792187 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:55.792255 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:55.792386 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:55.792387 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:55.792551 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:55.792598 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:55.792600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:55.793259 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:55.793260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:55.793778 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:55.793780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:55.794098 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:55.794099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:55.794136 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:55.794681 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:55.794686 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:55.796781 (+ 2095us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":113,"threads_started":1}
W20260924 21:12:55.796924 16134 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:55.792075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:38761 (local address 127.6.210.190:37673)
0924 21:12:55.792149 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:55.792151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:55.792264 (+ 113us) server_negotiation.cc:407] Connection header received
0924 21:12:55.792302 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:55.792302 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:55.792314 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:55.792370 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:55.792664 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:55.793144 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:55.793855 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:55.794033 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:55.794082 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:55.796865 (+ 2783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.131:38761: BlockingRecv error: recv got EOF from 127.6.210.131:38761 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 24 21:12:56 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:56 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:56.530822 16137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:56.525793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.129:60099)
0924 21:12:56.525942 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:56.525955 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:56.526031 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:56.526244 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:56.526245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:56.526417 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:56.526465 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:56.526467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:56.527249 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:56.527250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:56.527777 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:56.527778 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:56.528193 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:56.528193 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:56.528267 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:56.528699 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:56.528704 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:56.530718 (+ 2014us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":59,"threads_started":1}
W20260924 21:12:56.530997 16138 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:56.525889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:60099 (local address 127.6.210.190:37673)
0924 21:12:56.526052 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:56.526055 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:56.526108 (+ 53us) server_negotiation.cc:407] Connection header received
0924 21:12:56.526133 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:56.526133 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:56.526145 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:56.526213 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:56.526573 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:56.527149 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:56.527874 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:56.528115 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:56.528180 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:56.530902 (+ 2722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.129:60099: BlockingRecv error: recv got EOF from 127.6.210.129:60099 (error 108)
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":63,"threads_started":1}
Sep 24 21:12:56 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:56 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:56.661975 16139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:56.657297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.130:45253)
0924 21:12:56.657444 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:56.657465 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:56.657538 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:56.657688 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:56.657689 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:56.657871 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:56.657920 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:56.657922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:56.658684 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:56.658686 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:56.659261 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:56.659263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:56.659604 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:56.659605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:56.659649 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:56.660150 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:56.660155 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:56.661917 (+ 1762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":52,"threads_started":1}
W20260924 21:12:56.662122 16138 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:56.657403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:45253 (local address 127.6.210.190:37673)
0924 21:12:56.657494 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:56.657496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:56.657519 (+ 23us) server_negotiation.cc:407] Connection header received
0924 21:12:56.657601 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:56.657602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:56.657613 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:56.657664 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:56.658002 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:56.658589 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:56.659347 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:56.659531 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:56.659586 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:56.662043 (+ 2457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.130:45253: BlockingRecv error: recv got EOF from 127.6.210.130:45253 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 24 21:12:56 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:56 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:56.802176 16140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:56.797469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.131:46807)
0924 21:12:56.797614 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:56.797631 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:56.797686 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:56.797817 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:56.797818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:56.797973 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:56.798022 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:56.798024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:56.798701 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:56.798702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:56.799345 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:56.799347 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:56.799682 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:56.799683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:56.799736 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:56.800185 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:56.800190 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:56.802112 (+ 1922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":88,"threads_started":1}
W20260924 21:12:56.802325 16138 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:56.797529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:46807 (local address 127.6.210.190:37673)
0924 21:12:56.797605 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:56.797607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:56.797675 (+ 68us) server_negotiation.cc:407] Connection header received
0924 21:12:56.797733 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:56.797734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:56.797745 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:56.797798 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:56.798085 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:56.798605 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:56.799426 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:56.799596 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:56.799646 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:56.802196 (+ 2550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.131:46807: BlockingRecv error: recv got EOF from 127.6.210.131:46807 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 24 21:12:57 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:57 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:57.536140 16141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:57.531458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.129:59569)
0924 21:12:57.531607 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:57.531620 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:57.531687 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:57.531923 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:57.531924 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:57.532083 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:57.532129 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:57.532131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:57.532805 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:57.532807 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:57.533399 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:57.533402 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:57.533775 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:57.533776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:57.533826 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:57.534310 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:57.534314 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:57.536071 (+ 1757us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":59,"threads_started":1}
W20260924 21:12:57.536211 16142 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:57.531643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:59569 (local address 127.6.210.190:37673)
0924 21:12:57.531775 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:57.531777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:57.531785 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:12:57.531823 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:57.531824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:57.531834 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:57.531897 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:57.532198 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:57.532689 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:57.533480 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:57.533661 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:57.533710 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:57.536161 (+ 2451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.129:59569: BlockingRecv error: recv got EOF from 127.6.210.129:59569 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":61,"threads_started":1}
Sep 24 21:12:57 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:57 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:57.667363 16143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:57.662486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.130:37153)
0924 21:12:57.662633 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:57.662643 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:57.662696 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:57.662892 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:57.662894 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:57.663072 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:57.663146 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:57.663148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:57.663951 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:57.663952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:57.664540 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:57.664542 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:57.664899 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:57.664900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:57.664942 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:57.665397 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:57.665402 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:57.667287 (+ 1885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":72,"threads_started":1}
W20260924 21:12:57.667479 16142 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:57.662543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:37153 (local address 127.6.210.190:37673)
0924 21:12:57.662617 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:57.662619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:57.662744 (+ 125us) server_negotiation.cc:407] Connection header received
0924 21:12:57.662773 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:57.662773 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:57.662784 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:57.662829 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:57.663238 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:57.663866 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:57.664629 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:57.664829 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:57.664885 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:57.667403 (+ 2518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.130:37153: BlockingRecv error: recv got EOF from 127.6.210.130:37153 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 24 21:12:57 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:57 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:57.807832 16144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:57.802662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.131:48173)
0924 21:12:57.802854 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:57.802867 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:57.802970 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:57.803147 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:57.803148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:57.803388 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:57.803437 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:57.803439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:57.804231 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:57.804232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:57.804819 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:57.804820 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:57.805237 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:57.805238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:57.805308 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:57.805763 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:57.805768 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:57.807758 (+ 1990us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":110,"threads_started":1}
W20260924 21:12:57.807964 16142 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:57.802748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:48173 (local address 127.6.210.190:37673)
0924 21:12:57.802835 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:57.802837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:57.802943 (+ 106us) server_negotiation.cc:407] Connection header received
0924 21:12:57.803042 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:57.803043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:57.803059 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:57.803138 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:57.803528 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:57.804088 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:57.804910 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:57.805125 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:57.805237 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:57.807880 (+ 2643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.131:48173: BlockingRecv error: recv got EOF from 127.6.210.131:48173 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 24 21:12:58 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:58 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:58.542253 16145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:58.536729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.129:55441)
0924 21:12:58.536876 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:58.536895 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:58.536946 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:58.537160 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:58.537162 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:58.537401 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:58.537449 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:58.537451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:58.538131 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:58.538133 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:58.538875 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:58.538878 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:58.539275 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:58.539276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:58.539326 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:58.539911 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:58.539916 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:58.542152 (+ 2236us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":108,"threads_started":1}
W20260924 21:12:58.542464 16146 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:58.536798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:55441 (local address 127.6.210.190:37673)
0924 21:12:58.536947 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:58.536949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:58.536981 (+ 32us) server_negotiation.cc:407] Connection header received
0924 21:12:58.537003 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:58.537003 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:58.537013 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:58.537066 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:58.537511 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:58.538024 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:58.538963 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:58.539157 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:58.539216 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:58.542375 (+ 3159us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.129:55441: BlockingRecv error: recv got EOF from 127.6.210.129:55441 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":94,"threads_started":1}
Sep 24 21:12:58 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:58 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:58.672945 16147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:58.667922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.130:45533)
0924 21:12:58.668045 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:58.668065 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:58.668118 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:58.668297 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:58.668298 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:58.668472 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:58.668539 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:58.668540 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:58.669293 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:58.669294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:58.669881 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:58.669883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:58.670385 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:58.670386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:58.670434 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:58.670878 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:58.670883 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:58.672884 (+ 2001us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":67,"threads_started":1}
W20260924 21:12:58.673062 16146 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:58.668032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:45533 (local address 127.6.210.190:37673)
0924 21:12:58.668116 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:58.668118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:58.668131 (+ 13us) server_negotiation.cc:407] Connection header received
0924 21:12:58.668202 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:58.668203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:58.668214 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:58.668282 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:58.668661 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:58.669158 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:58.669981 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:58.670175 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:58.670302 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:58.672984 (+ 2682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.130:45533: BlockingRecv error: recv got EOF from 127.6.210.130:45533 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 24 21:12:58 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:58 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:58.812877 16148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:58.808330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.131:34023)
0924 21:12:58.808469 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:58.808481 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:58.808546 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:58.808670 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:58.808671 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:58.808854 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:58.808902 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:58.808903 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:58.809573 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:58.809574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:58.810096 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:58.810097 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:58.810447 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:58.810447 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:58.810489 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:58.810968 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:58.810973 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:58.812780 (+ 1807us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":84,"threads_started":1}
W20260924 21:12:58.812916 16146 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:58.808415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:34023 (local address 127.6.210.190:37673)
0924 21:12:58.808475 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:58.808476 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:58.808537 (+ 61us) server_negotiation.cc:407] Connection header received
0924 21:12:58.808587 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:58.808588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:58.808598 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:58.808651 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:58.808966 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:58.809499 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:58.810168 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:58.810387 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:58.810439 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:58.812856 (+ 2417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.131:34023: BlockingRecv error: recv got EOF from 127.6.210.131:34023 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 24 21:12:59 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:59 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:59.547650 16149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:59.542830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.129:44409)
0924 21:12:59.542956 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:59.542966 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:59.543023 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:59.543175 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:59.543176 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:59.543341 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:59.543388 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:59.543390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:59.544159 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:59.544161 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:59.544805 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:59.544808 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:59.545132 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:59.545132 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:59.545178 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:59.545672 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:59.545677 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:59.547579 (+ 1902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":77,"threads_started":1}
W20260924 21:12:59.547782 16150 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:59.542924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:44409 (local address 127.6.210.190:37673)
0924 21:12:59.543024 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:59.543026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:59.543063 (+ 37us) server_negotiation.cc:407] Connection header received
0924 21:12:59.543085 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:59.543085 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:59.543096 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:59.543145 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:59.543457 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:59.544077 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:59.544887 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:59.545066 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:59.545115 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:59.547701 (+ 2586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.129:44409: BlockingRecv error: recv got EOF from 127.6.210.129:44409 (error 108)
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":47,"threads_started":1}
Sep 24 21:12:59 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:59 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:59.678558 16151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:59.673480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.130:35381)
0924 21:12:59.673662 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:59.673673 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:59.673729 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:59.673867 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:59.673867 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:59.674026 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:59.674074 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:59.674075 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:59.674832 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:59.674833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:59.675469 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:59.675471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:59.675834 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:59.675835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:59.675883 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:59.676426 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:59.676431 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:59.678501 (+ 2070us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":98,"threads_started":1}
W20260924 21:12:59.678683 16150 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:59.673628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:35381 (local address 127.6.210.190:37673)
0924 21:12:59.673706 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:59.673708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:59.673717 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:12:59.673786 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:59.673787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:59.673796 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:59.673847 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:59.674217 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:59.674740 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:59.675567 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:59.675758 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:59.675816 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:59.678591 (+ 2775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.130:35381: BlockingRecv error: recv got EOF from 127.6.210.130:35381 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 24 21:12:59 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:12:59 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:12:59.821843 16152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:12:59.813345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.131:35407)
0924 21:12:59.813477 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0924 21:12:59.813490 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:12:59.813540 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:12:59.813674 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:12:59.813675 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:12:59.813837 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:12:59.813883 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:59.813885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:59.814594 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:59.814595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:59.815119 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:12:59.815121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:12:59.815467 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:12:59.815468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:12:59.815535 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:59.819744 (+ 4209us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:12:59.819749 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:12:59.821743 (+ 1994us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":74,"threads_started":1}
W20260924 21:12:59.821897 16150 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:12:59.813488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:35407 (local address 127.6.210.190:37673)
0924 21:12:59.813545 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 21:12:59.813547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:12:59.813580 (+ 33us) server_negotiation.cc:407] Connection header received
0924 21:12:59.813601 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:12:59.813602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:12:59.813612 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:12:59.813658 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:12:59.813948 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:59.814519 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:59.815212 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:12:59.815402 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:12:59.815459 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:12:59.821830 (+ 6371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.131:35407: BlockingRecv error: recv got EOF from 127.6.210.131:35407 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Sep 24 21:13:00 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:13:00 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:13:00.553182 16153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:00.548094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.129:40927)
0924 21:13:00.548253 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:00.548274 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:00.548324 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:00.548487 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:00.548488 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:00.548665 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:13:00.548711 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:00.548712 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:00.549468 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:00.549470 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:00.550065 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:00.550067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:00.550541 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:00.550542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:00.550592 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:00.551221 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:13:00.551226 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:13:00.553093 (+ 1867us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":103,"threads_started":1}
W20260924 21:13:00.553299 16154 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:13:00.548201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:40927 (local address 127.6.210.190:37673)
0924 21:13:00.548337 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:00.548339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:00.548355 (+ 16us) server_negotiation.cc:407] Connection header received
0924 21:13:00.548385 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:00.548385 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:00.548399 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:00.548470 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:00.548796 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:00.549377 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:00.550204 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:00.550458 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:00.550524 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:00.553243 (+ 2719us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.129:40927: BlockingRecv error: recv got EOF from 127.6.210.129:40927 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
Sep 24 21:13:00 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:13:00 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:13:00.684194 16155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:00.679098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.130:42315)
0924 21:13:00.679275 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:00.679287 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:00.679343 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:00.679461 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:00.679461 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:00.679624 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:13:00.679668 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:00.679669 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:00.680382 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:00.680383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:00.680992 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:00.680994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:00.681424 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:00.681424 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:00.681468 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:00.682008 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:13:00.682016 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:13:00.684127 (+ 2111us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":66,"threads_started":1}
W20260924 21:13:00.684325 16154 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:13:00.679266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:42315 (local address 127.6.210.190:37673)
0924 21:13:00.679343 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:00.679345 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:00.679355 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:13:00.679385 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:00.679386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:00.679396 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:00.679455 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:00.679733 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:00.680312 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:00.681073 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:00.681348 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:00.681410 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:00.684246 (+ 2836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.130:42315: BlockingRecv error: recv got EOF from 127.6.210.130:42315 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 24 21:13:00 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:13:00 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:13:00.827376 16156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:00.822425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.131:41055)
0924 21:13:00.822617 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:00.822631 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:00.822717 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:00.822892 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:00.822893 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:00.823074 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:13:00.823121 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:00.823122 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:00.823780 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:00.823781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:00.824431 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:00.824433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:00.824782 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:00.824783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:00.824855 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:00.825335 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:13:00.825340 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:13:00.827302 (+ 1962us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":88,"threads_started":1}
W20260924 21:13:00.827540 16154 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:13:00.822511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:41055 (local address 127.6.210.190:37673)
0924 21:13:00.822592 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:00.822594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:00.822708 (+ 114us) server_negotiation.cc:407] Connection header received
0924 21:13:00.822781 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:00.822782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:00.822793 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:00.822841 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:00.823220 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:00.823699 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:00.824513 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:00.824693 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:00.824765 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:00.827436 (+ 2671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.131:41055: BlockingRecv error: recv got EOF from 127.6.210.131:41055 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 24 21:13:01 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:13:01 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:13:01.558609 16157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:01.553796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.129:56609)
0924 21:13:01.553931 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:01.553946 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:01.554025 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:01.554214 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:01.554215 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:01.554447 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:13:01.554491 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:01.554493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:01.555137 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:01.555138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:01.555674 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:01.555675 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:01.556028 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:01.556029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:01.556074 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:01.556593 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:13:01.556598 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:13:01.558550 (+ 1952us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":69,"threads_started":1}
W20260924 21:13:01.558710 16158 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:13:01.553854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:56609 (local address 127.6.210.190:37673)
0924 21:13:01.553972 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:01.553976 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:01.553990 (+ 14us) server_negotiation.cc:407] Connection header received
0924 21:13:01.554104 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:01.554105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:01.554117 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:01.554184 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:01.554565 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:01.555051 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:01.555773 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:01.555959 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:01.556013 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:01.558639 (+ 2626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.129:56609: BlockingRecv error: recv got EOF from 127.6.210.129:56609 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":55,"threads_started":1}
Sep 24 21:13:01 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:13:01 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:13:01.689122 16159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:01.684780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.130:38043)
0924 21:13:01.684912 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:01.684925 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:01.684992 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:01.685101 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:01.685102 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:01.685266 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:13:01.685312 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:01.685314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:01.685926 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:01.685927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:01.686472 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:01.686474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:01.686858 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:01.686858 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:01.686896 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:01.687349 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:13:01.687354 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:13:01.689063 (+ 1709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
W20260924 21:13:01.689313 16158 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:13:01.684844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:38043 (local address 127.6.210.190:37673)
0924 21:13:01.684888 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:01.684891 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:01.684979 (+ 88us) server_negotiation.cc:407] Connection header received
0924 21:13:01.685047 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:01.685048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:01.685057 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:01.685096 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:01.685377 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:01.685858 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:01.686566 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:01.686781 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:01.686852 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:01.689178 (+ 2326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.130:38043: BlockingRecv error: recv got EOF from 127.6.210.130:38043 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Sep 24 21:13:01 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:13:01 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:13:01.832335 16160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:01.827906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.131:57479)
0924 21:13:01.828055 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:01.828069 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:01.828122 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:01.828275 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:01.828276 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:01.828438 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:13:01.828481 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:01.828482 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:01.829106 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:01.829107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:01.829644 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:01.829646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:01.829991 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:01.829992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:01.830029 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:01.830501 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:13:01.830506 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:13:01.832272 (+ 1766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":97,"threads_started":1}
W20260924 21:13:01.832463 16158 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:13:01.827963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:57479 (local address 127.6.210.190:37673)
0924 21:13:01.828033 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:01.828035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:01.828125 (+ 90us) server_negotiation.cc:407] Connection header received
0924 21:13:01.828193 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:01.828193 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:01.828205 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:01.828257 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:01.828549 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:01.829043 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:01.829731 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:01.829923 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:01.829980 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:01.832378 (+ 2398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.131:57479: BlockingRecv error: recv got EOF from 127.6.210.131:57479 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 24 21:13:02 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:13:02 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:13:02.564698 16161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:02.559244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.129:46713)
0924 21:13:02.559476 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:02.559489 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:02.559562 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:02.559747 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:02.559748 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:02.559918 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:13:02.559965 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:02.559967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:02.560772 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:02.560773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:02.561577 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:02.561580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:02.561960 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:02.561960 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:02.562004 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:02.562621 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:13:02.562626 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:13:02.564599 (+ 1973us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":181,"threads_started":1}
W20260924 21:13:02.564774 16162 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:13:02.559365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:46713 (local address 127.6.210.190:37673)
0924 21:13:02.559506 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:02.559508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:02.559545 (+ 37us) server_negotiation.cc:407] Connection header received
0924 21:13:02.559647 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:02.559647 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:02.559659 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:02.559729 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:02.560045 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:02.560677 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:02.561648 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:02.561886 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:02.561949 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:02.564717 (+ 2768us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.129:46713: BlockingRecv error: recv got EOF from 127.6.210.129:46713 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
Sep 24 21:13:02 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:13:02 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:13:02.694957 16163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:02.689786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.130:52691)
0924 21:13:02.689944 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:02.689955 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:02.690018 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:02.690241 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:02.690242 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:02.690462 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:13:02.690507 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:02.690509 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:02.691289 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:02.691290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:02.691818 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:02.691819 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:02.692215 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:02.692215 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:02.692290 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:02.692996 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:13:02.693001 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:13:02.694887 (+ 1886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":83,"threads_started":1}
W20260924 21:13:02.695111 16162 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:13:02.689986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:52691 (local address 127.6.210.190:37673)
0924 21:13:02.690063 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:02.690065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:02.690073 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:13:02.690104 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:02.690105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:02.690115 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:02.690183 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:02.690613 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:02.691177 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:02.691898 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:02.692095 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:02.692198 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:02.695021 (+ 2823us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.130:52691: BlockingRecv error: recv got EOF from 127.6.210.130:52691 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 24 21:13:02 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:13:02 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:13:02.837623 16164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:02.833022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.131:39865)
0924 21:13:02.833165 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:02.833178 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:02.833247 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:02.833399 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:02.833400 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:02.833577 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:13:02.833620 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:02.833622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:02.834352 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:02.834353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:02.834905 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:02.834907 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:02.835272 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:02.835272 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:02.835311 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:02.835790 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:13:02.835794 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:13:02.837557 (+ 1763us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":72,"threads_started":1}
W20260924 21:13:02.837759 16162 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:13:02.833063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:39865 (local address 127.6.210.190:37673)
0924 21:13:02.833132 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:02.833134 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:02.833249 (+ 115us) server_negotiation.cc:407] Connection header received
0924 21:13:02.833305 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:02.833306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:02.833320 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:02.833382 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:02.833689 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:02.834224 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:02.834981 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:02.835204 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:02.835261 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:02.837656 (+ 2395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.131:39865: BlockingRecv error: recv got EOF from 127.6.210.131:39865 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 24 21:13:03 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:13:03 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:13:03.570135 16165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:03.565284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.129:53823)
0924 21:13:03.565448 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:03.565465 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:03.565533 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:03.565679 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:03.565680 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:03.565850 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:13:03.565897 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:03.565899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:03.566600 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:03.566601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:03.567229 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:03.567232 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:03.567572 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:03.567573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:03.567616 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:03.568080 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:13:03.568085 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:13:03.570067 (+ 1982us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":108,"threads_started":1}
W20260924 21:13:03.570207 16166 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:13:03.565337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:53823 (local address 127.6.210.190:37673)
0924 21:13:03.565465 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:03.565468 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:03.565520 (+ 52us) server_negotiation.cc:407] Connection header received
0924 21:13:03.565578 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:03.565578 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:03.565588 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:03.565650 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:03.565967 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:03.566497 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:03.567318 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:03.567506 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:03.567557 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:03.570155 (+ 2598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.129:53823: BlockingRecv error: recv got EOF from 127.6.210.129:53823 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":75,"threads_started":1}
Sep 24 21:13:03 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:13:03 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:13:03.699862 16167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:03.695414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.130:50019)
0924 21:13:03.695553 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:03.695564 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:03.695621 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:03.695816 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:03.695817 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:03.695997 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:13:03.696045 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:03.696047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:03.696671 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:03.696672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:03.697209 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:03.697211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:03.697521 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:03.697522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:03.697561 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:03.698057 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:13:03.698062 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:13:03.699805 (+ 1743us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
W20260924 21:13:03.700004 16166 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:13:03.695610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:50019 (local address 127.6.210.190:37673)
0924 21:13:03.695685 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:03.695686 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:03.695695 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:13:03.695727 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:03.695728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:03.695738 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:03.695792 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:03.696112 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:03.696600 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:03.697281 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:03.697461 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:03.697509 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:03.699923 (+ 2414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.130:50019: BlockingRecv error: recv got EOF from 127.6.210.130:50019 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 24 21:13:03 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:13:03 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.131@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:13:03.842914 16168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:03.838132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.131:53687)
0924 21:13:03.838307 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:03.838318 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:03.838375 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:03.838490 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:03.838491 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:03.838653 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:13:03.838699 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:03.838700 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:03.839406 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:03.839407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:03.839948 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:03.839950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:03.840288 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:03.840289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:03.840325 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:03.840810 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:13:03.840814 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:13:03.842844 (+ 2030us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":83,"threads_started":1}
W20260924 21:13:03.842985 16166 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:13:03.838177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:53687 (local address 127.6.210.190:37673)
0924 21:13:03.838224 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:03.838226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:03.838383 (+ 157us) server_negotiation.cc:407] Connection header received
0924 21:13:03.838412 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:03.838412 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:03.838425 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:03.838475 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:03.838763 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:03.839340 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:03.840014 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:03.840225 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:03.840276 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:03.842940 (+ 2664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.131:53687: BlockingRecv error: recv got EOF from 127.6.210.131:53687 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Sep 24 21:13:04 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:13:04 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.129@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:13:04.576303 16169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:04.570912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.129:42217)
0924 21:13:04.571381 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:04.571416 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:04.571491 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:04.571638 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:04.571639 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:04.571799 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:13:04.571846 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:04.571848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:04.572474 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:04.572475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:04.573068 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:04.573069 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:04.573419 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:04.573420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:04.573464 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:04.573985 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:13:04.573990 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:13:04.576141 (+ 2151us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":74,"threads_started":1}
W20260924 21:13:04.576306 16170 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:13:04.570997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:42217 (local address 127.6.210.190:37673)
0924 21:13:04.571116 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:04.571118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:04.571497 (+ 379us) server_negotiation.cc:407] Connection header received
0924 21:13:04.571554 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:04.571555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:04.571567 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:04.571620 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:04.571912 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:04.572405 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:04.573150 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:04.573349 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:04.573403 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:04.576233 (+ 2830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.129:42217: BlockingRecv error: recv got EOF from 127.6.210.129:42217 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":53,"threads_started":1}
Sep 24 21:13:04 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:13:04 dist-test-slave-q276 krb5kdc[15111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.210.130@KRBTEST.COM for oryx/127.6.210.190@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:13:04.705768 16171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:04.700455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37673 (local address 127.6.210.130:55463)
0924 21:13:04.700650 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:04.700663 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:04.700739 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:04.700914 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:04.700915 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:04.701156 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:13:04.701242 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:04.701244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:04.701994 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:04.701995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:04.702648 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:04.702651 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:04.703081 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:04.703082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:04.703155 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:04.703714 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:13:04.703719 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:13:04.705653 (+ 1934us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.210.190:37673: Server oryx/127.6.210.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":109,"threads_started":1}
W20260924 21:13:04.705806 16170 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:13:04.700562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:55463 (local address 127.6.210.190:37673)
0924 21:13:04.700621 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:04.700623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:04.700784 (+ 161us) server_negotiation.cc:407] Connection header received
0924 21:13:04.700815 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:04.700816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:04.700829 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:04.700891 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:04.701330 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:04.701900 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:04.702742 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:04.702982 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:04.703082 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:04.705737 (+ 2655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.210.130:55463: BlockingRecv error: recv got EOF from 127.6.210.130:55463 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 21:13:04.744396 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 15694
I20260924 21:13:04.748965 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 15846
I20260924 21:13:04.753096 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 15980
I20260924 21:13:04.757236 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 15624
2026-09-24T21:13:04Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17466 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-24T21:13:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:13:04Z Disabled control of system clock
I20260924 21:13:04.784363 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:39271
--webserver_interface=127.6.210.190
--webserver_port=0
--builtin_ntp_servers=127.6.210.148:36415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:39271
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260924 21:13:04.863054 16175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:13:04.863255 16175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:13:04.863281 16175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:13:04.864462 16175 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:13:04.864503 16175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:13:04.864516 16175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:13:04.864528 16175 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:13:04.864540 16175 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:13:04.865882 16175 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:36415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:39271
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:39271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:13:04.866133 16175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:13:04.866449 16175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:13:04.871476 16180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:13:04.871470 16183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:13:04.871470 16181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:13:04.871591 16175 server_base.cc:1034] running on GCE node
I20260924 21:13:04.872056 16175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:13:04.872354 16175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:13:04.873518 16175 hybrid_clock.cc:648] HybridClock initialized: now 1790284384873512 us; error 36 us; skew 500 ppm
I20260924 21:13:04.879812 16175 webserver.cc:460] Webserver started at http://127.6.210.190:39131/ using document root <none> and password file <none>
I20260924 21:13:04.880123 16175 fs_manager.cc:359] Metadata directory not provided
I20260924 21:13:04.880221 16175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:13:04.880388 16175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:13:04.881597 16175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "9edc249c08094e8d990774142923ed26"
format_stamp: "Formatted at 2026-09-24 21:13:04 on dist-test-slave-q276"
server_key: "67b209f488194732d151495a31577769"
server_key_iv: "565259c40d65e729693cd349a9bfef65"
server_key_version: "encryptionkey@0"
I20260924 21:13:04.882002 16175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "9edc249c08094e8d990774142923ed26"
format_stamp: "Formatted at 2026-09-24 21:13:04 on dist-test-slave-q276"
server_key: "67b209f488194732d151495a31577769"
server_key_iv: "565259c40d65e729693cd349a9bfef65"
server_key_version: "encryptionkey@0"
I20260924 21:13:04.883522 16175 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:13:04.884452 16189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:04.884649 16175 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:13:04.884783 16175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "9edc249c08094e8d990774142923ed26"
format_stamp: "Formatted at 2026-09-24 21:13:04 on dist-test-slave-q276"
server_key: "67b209f488194732d151495a31577769"
server_key_iv: "565259c40d65e729693cd349a9bfef65"
server_key_version: "encryptionkey@0"
I20260924 21:13:04.884871 16175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:13:04.922859 16175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:13:04.923249 16175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:13:04.923429 16175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:13:04.927764 16175 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:39271
I20260924 21:13:04.927800 16241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:39271 every 8 connection(s)
I20260924 21:13:04.928184 16175 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:13:04.928817 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 16175
I20260924 21:13:04.928836 16242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:13:04.928936 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:13:04.929147 6986 external_mini_cluster.cc:1428] Setting key 4d9823dea2336d18fb7b63701b7d5d43
I20260924 21:13:04.932438 16242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9edc249c08094e8d990774142923ed26: Bootstrap starting.
I20260924 21:13:04.933100 16242 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9edc249c08094e8d990774142923ed26: Neither blocks nor log segments found. Creating new log.
I20260924 21:13:04.933386 16242 log.cc:824] T 00000000000000000000000000000000 P 9edc249c08094e8d990774142923ed26: Log is configured to *not* fsync() on all Append() calls
I20260924 21:13:04.934063 16242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9edc249c08094e8d990774142923ed26: No bootstrap required, opened a new log
I20260924 21:13:04.935046 16245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:04.929756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54454 (local address 127.6.210.190:39271)
0924 21:13:04.929961 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:04.929969 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:04.929995 (+ 26us) server_negotiation.cc:407] Connection header received
0924 21:13:04.930483 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:04.930491 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:04.930574 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:04.930708 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:04.930928 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:04.931849 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:04.932937 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:04.933429 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:04.933585 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:04.934380 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:13:04.934384 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:13:04.934384 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:13:04.934564 (+ 180us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:13:04.934602 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:13:04.934613 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:13:04.934697 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0924 21:13:04.934785 (+ 88us) server_negotiation.cc:299] Negotiation successful
0924 21:13:04.934871 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":81,"threads_started":1}
I20260924 21:13:04.936090 16242 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9edc249c08094e8d990774142923ed26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9edc249c08094e8d990774142923ed26" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39271 } }
I20260924 21:13:04.936241 16242 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9edc249c08094e8d990774142923ed26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:13:04.936324 16242 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9edc249c08094e8d990774142923ed26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9edc249c08094e8d990774142923ed26, State: Initialized, Role: FOLLOWER
I20260924 21:13:04.936468 16242 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9edc249c08094e8d990774142923ed26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9edc249c08094e8d990774142923ed26" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39271 } }
I20260924 21:13:04.936580 16242 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9edc249c08094e8d990774142923ed26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:13:04.936625 16242 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9edc249c08094e8d990774142923ed26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:13:04.936672 16242 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9edc249c08094e8d990774142923ed26 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:13:04.937238 16242 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9edc249c08094e8d990774142923ed26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9edc249c08094e8d990774142923ed26" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39271 } }
I20260924 21:13:04.937361 16242 leader_election.cc:302] T 00000000000000000000000000000000 P 9edc249c08094e8d990774142923ed26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9edc249c08094e8d990774142923ed26; no voters:
I20260924 21:13:04.937513 16242 leader_election.cc:288] T 00000000000000000000000000000000 P 9edc249c08094e8d990774142923ed26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:13:04.937568 16247 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9edc249c08094e8d990774142923ed26 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:13:04.937769 16247 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9edc249c08094e8d990774142923ed26 [term 1 LEADER]: Becoming Leader. State: Replica: 9edc249c08094e8d990774142923ed26, State: Running, Role: LEADER
I20260924 21:13:04.937799 16242 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9edc249c08094e8d990774142923ed26 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:13:04.937881 16247 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9edc249c08094e8d990774142923ed26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9edc249c08094e8d990774142923ed26" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39271 } }
I20260924 21:13:04.938213 16247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9edc249c08094e8d990774142923ed26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9edc249c08094e8d990774142923ed26. Latest consensus state: current_term: 1 leader_uuid: "9edc249c08094e8d990774142923ed26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9edc249c08094e8d990774142923ed26" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39271 } } }
I20260924 21:13:04.938408 16247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9edc249c08094e8d990774142923ed26 [sys.catalog]: This master's current role is: LEADER
I20260924 21:13:04.938225 16249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9edc249c08094e8d990774142923ed26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9edc249c08094e8d990774142923ed26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9edc249c08094e8d990774142923ed26" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39271 } } }
I20260924 21:13:04.938517 16249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9edc249c08094e8d990774142923ed26 [sys.catalog]: This master's current role is: LEADER
I20260924 21:13:04.938711 16256 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:13:04.939221 16256 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:13:04.940891 16256 catalog_manager.cc:1347] Generated new cluster ID: 2757ed74201a4ca69e14ee1252476f36
I20260924 21:13:04.940944 16256 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:13:04.957327 16256 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:13:04.958118 16256 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:13:04.979568 16256 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9edc249c08094e8d990774142923ed26: Generated new TSK 0
I20260924 21:13:04.979801 16256 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:13:04.985327 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.129:0
--local_ip_for_outbound_sockets=127.6.210.129
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:39271
--builtin_ntp_servers=127.6.210.148:36415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260924 21:13:05.074779 16266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:13:05.074985 16266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:13:05.075006 16266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:13:05.076191 16266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:13:05.076233 16266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:13:05.076264 16266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.129
I20260924 21:13:05.077611 16266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:36415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:39271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.129
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:13:05.077872 16266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:13:05.078171 16266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:13:05.083402 16274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:13:05.083426 16271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:13:05.083429 16272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:13:05.083678 16266 server_base.cc:1034] running on GCE node
I20260924 21:13:05.083904 16266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:13:05.084102 16266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:13:05.085268 16266 hybrid_clock.cc:648] HybridClock initialized: now 1790284385085254 us; error 45 us; skew 500 ppm
I20260924 21:13:05.086658 16266 webserver.cc:460] Webserver started at http://127.6.210.129:38797/ using document root <none> and password file <none>
I20260924 21:13:05.086927 16266 fs_manager.cc:359] Metadata directory not provided
I20260924 21:13:05.086999 16266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:13:05.087144 16266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:13:05.088300 16266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/data/instance:
uuid: "0b863a3c013e41bebf46a152e5c01aa0"
format_stamp: "Formatted at 2026-09-24 21:13:05 on dist-test-slave-q276"
server_key: "49976cd1b261e6dfe84744e256039a47"
server_key_iv: "dcdf228c070ccc7b68362cea063e0856"
server_key_version: "encryptionkey@0"
I20260924 21:13:05.088663 16266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance:
uuid: "0b863a3c013e41bebf46a152e5c01aa0"
format_stamp: "Formatted at 2026-09-24 21:13:05 on dist-test-slave-q276"
server_key: "49976cd1b261e6dfe84744e256039a47"
server_key_iv: "dcdf228c070ccc7b68362cea063e0856"
server_key_version: "encryptionkey@0"
I20260924 21:13:05.090076 16266 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:13:05.090979 16280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:05.091159 16266 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:13:05.091259 16266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/wal
uuid: "0b863a3c013e41bebf46a152e5c01aa0"
format_stamp: "Formatted at 2026-09-24 21:13:05 on dist-test-slave-q276"
server_key: "49976cd1b261e6dfe84744e256039a47"
server_key_iv: "dcdf228c070ccc7b68362cea063e0856"
server_key_version: "encryptionkey@0"
I20260924 21:13:05.091346 16266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:13:05.099220 16266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:13:05.099485 16266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:13:05.099627 16266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:13:05.099848 16266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:13:05.100147 16266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:13:05.100199 16266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:05.100238 16266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:13:05.100272 16266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:05.106029 16266 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.129:46093
I20260924 21:13:05.106076 16393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.129:46093 every 8 connection(s)
I20260924 21:13:05.106454 16266 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
I20260924 21:13:05.110536 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 16266
I20260924 21:13:05.110633 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance
I20260924 21:13:05.110827 6986 external_mini_cluster.cc:1428] Setting key 63bd46fb984bccf5c26d6ec87c29b06d
I20260924 21:13:05.112143 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.130:0
--local_ip_for_outbound_sockets=127.6.210.130
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:39271
--builtin_ntp_servers=127.6.210.148:36415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260924 21:13:05.112268 16245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:05.107156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:41659 (local address 127.6.210.190:39271)
0924 21:13:05.107205 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:05.107207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:05.107451 (+ 244us) server_negotiation.cc:407] Connection header received
0924 21:13:05.107952 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:05.107953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:05.107966 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:05.108020 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:05.108699 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:05.109179 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:05.110160 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:05.110447 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:05.110525 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:05.111809 (+ 1284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:13:05.111810 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:13:05.111810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:13:05.111816 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:13:05.111834 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:13:05.111839 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:13:05.111904 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 21:13:05.112017 (+ 113us) server_negotiation.cc:299] Negotiation successful
0924 21:13:05.112060 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 21:13:05.112260 16396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:05.107133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39271 (local address 127.6.210.129:41659)
0924 21:13:05.107316 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:05.107350 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:05.107863 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:05.108076 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:05.108080 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:05.108088 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:13:05.108624 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:05.108628 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:05.109285 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:05.109286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:05.110080 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:05.110082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:05.110544 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:05.110545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:05.110736 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:05.111477 (+ 741us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:13:05.111477 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:13:05.111505 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:13:05.111564 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:13:05.111925 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:13:05.111928 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:13:05.111931 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:13:05.111986 (+ 55us) client_negotiation.cc:240] Negotiation successful
0924 21:13:05.112069 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
I20260924 21:13:05.112813 16394 heartbeater.cc:347] Connected to a master server at 127.6.210.190:39271
I20260924 21:13:05.112903 16394 heartbeater.cc:464] Registering TS with master...
I20260924 21:13:05.113085 16394 heartbeater.cc:511] Master 127.6.210.190:39271 requested a full tablet report, sending...
I20260924 21:13:05.113652 16206 ts_manager.cc:194] Registered new tserver with Master: 0b863a3c013e41bebf46a152e5c01aa0 (127.6.210.129:46093)
I20260924 21:13:05.114421 16206 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.210.129:41659
W20260924 21:13:05.192224 16397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:13:05.192428 16397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:13:05.192451 16397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:13:05.193601 16397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:13:05.193686 16397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:13:05.193729 16397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.130
I20260924 21:13:05.195266 16397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:36415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:39271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.130
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:13:05.195515 16397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:13:05.195806 16397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:13:05.200727 16402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:13:05.200784 16403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:13:05.200739 16405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:13:05.201032 16397 server_base.cc:1034] running on GCE node
I20260924 21:13:05.201256 16397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:13:05.201534 16397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:13:05.202667 16397 hybrid_clock.cc:648] HybridClock initialized: now 1790284385202657 us; error 30 us; skew 500 ppm
I20260924 21:13:05.203871 16397 webserver.cc:460] Webserver started at http://127.6.210.130:38295/ using document root <none> and password file <none>
I20260924 21:13:05.204116 16397 fs_manager.cc:359] Metadata directory not provided
I20260924 21:13:05.204185 16397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:13:05.204339 16397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:13:05.205385 16397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/data/instance:
uuid: "e5fb7f8a660b43fe83057e5dd13ecfca"
format_stamp: "Formatted at 2026-09-24 21:13:05 on dist-test-slave-q276"
server_key: "5b307854b56b7821d4b6c14c7c0b2695"
server_key_iv: "2fcbd2679df455500f6cb0431ab4ff9f"
server_key_version: "encryptionkey@0"
I20260924 21:13:05.205730 16397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance:
uuid: "e5fb7f8a660b43fe83057e5dd13ecfca"
format_stamp: "Formatted at 2026-09-24 21:13:05 on dist-test-slave-q276"
server_key: "5b307854b56b7821d4b6c14c7c0b2695"
server_key_iv: "2fcbd2679df455500f6cb0431ab4ff9f"
server_key_version: "encryptionkey@0"
I20260924 21:13:05.207656 16397 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260924 21:13:05.208457 16411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:05.208631 16397 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:13:05.208726 16397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/wal
uuid: "e5fb7f8a660b43fe83057e5dd13ecfca"
format_stamp: "Formatted at 2026-09-24 21:13:05 on dist-test-slave-q276"
server_key: "5b307854b56b7821d4b6c14c7c0b2695"
server_key_iv: "2fcbd2679df455500f6cb0431ab4ff9f"
server_key_version: "encryptionkey@0"
I20260924 21:13:05.208813 16397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:13:05.223416 16397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:13:05.223713 16397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:13:05.223862 16397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:13:05.224087 16397 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:13:05.224407 16397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:13:05.224462 16397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:05.224501 16397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:13:05.224530 16397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:05.230232 16397 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.130:44021
I20260924 21:13:05.230293 16524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.130:44021 every 8 connection(s)
I20260924 21:13:05.230607 16397 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
I20260924 21:13:05.235760 16245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:05.231288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:34525 (local address 127.6.210.190:39271)
0924 21:13:05.231354 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:05.231356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:05.231632 (+ 276us) server_negotiation.cc:407] Connection header received
0924 21:13:05.232151 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:05.232152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:05.232163 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:05.232236 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:05.232876 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:05.233418 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:05.234508 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:05.234716 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:05.234779 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:05.235529 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:13:05.235530 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:13:05.235531 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:13:05.235535 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:13:05.235549 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:13:05.235553 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:13:05.235607 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 21:13:05.235660 (+ 53us) server_negotiation.cc:299] Negotiation successful
0924 21:13:05.235698 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 21:13:05.235875 16527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:05.231287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39271 (local address 127.6.210.130:34525)
0924 21:13:05.231481 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:05.231513 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:05.232046 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:05.232297 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:05.232300 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:05.232308 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:13:05.232790 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:05.232794 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:05.233498 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:05.233499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:05.234255 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:05.234257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:05.234790 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:05.234791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:05.234898 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:05.235385 (+ 487us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:13:05.235385 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:13:05.235412 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:13:05.235434 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:13:05.235609 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:13:05.235611 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:13:05.235613 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 21:13:05.235654 (+ 41us) client_negotiation.cc:240] Negotiation successful
0924 21:13:05.235730 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":91,"threads_started":1}
I20260924 21:13:05.236436 16525 heartbeater.cc:347] Connected to a master server at 127.6.210.190:39271
I20260924 21:13:05.236554 16525 heartbeater.cc:464] Registering TS with master...
I20260924 21:13:05.236778 16525 heartbeater.cc:511] Master 127.6.210.190:39271 requested a full tablet report, sending...
I20260924 21:13:05.237249 16206 ts_manager.cc:194] Registered new tserver with Master: e5fb7f8a660b43fe83057e5dd13ecfca (127.6.210.130:44021)
I20260924 21:13:05.237283 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 16397
I20260924 21:13:05.237370 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance
I20260924 21:13:05.237569 6986 external_mini_cluster.cc:1428] Setting key 711a527e9f41520bfe9ceb6656210cbf
I20260924 21:13:05.237712 16206 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.210.130:34525
I20260924 21:13:05.238658 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.131:0
--local_ip_for_outbound_sockets=127.6.210.131
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:39271
--builtin_ntp_servers=127.6.210.148:36415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260924 21:13:05.318056 16528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:13:05.318322 16528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:13:05.318359 16528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:13:05.319511 16528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:13:05.319576 16528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:13:05.319615 16528 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.131
I20260924 21:13:05.320956 16528 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:36415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:39271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.131
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:13:05.321211 16528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:13:05.321504 16528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:13:05.326613 16534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:13:05.326625 16536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:13:05.326669 16533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:13:05.326689 16528 server_base.cc:1034] running on GCE node
I20260924 21:13:05.327066 16528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:13:05.327325 16528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:13:05.328485 16528 hybrid_clock.cc:648] HybridClock initialized: now 1790284385328486 us; error 42 us; skew 500 ppm
I20260924 21:13:05.329723 16528 webserver.cc:460] Webserver started at http://127.6.210.131:37321/ using document root <none> and password file <none>
I20260924 21:13:05.329977 16528 fs_manager.cc:359] Metadata directory not provided
I20260924 21:13:05.330045 16528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:13:05.330204 16528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:13:05.331306 16528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/data/instance:
uuid: "fcb0892c755540c68dc6216f2b5999ec"
format_stamp: "Formatted at 2026-09-24 21:13:05 on dist-test-slave-q276"
server_key: "4e16e65ffe110cdd43d700e04eab6b10"
server_key_iv: "a6947b6bd7f092db79b483b68aabafb8"
server_key_version: "encryptionkey@0"
I20260924 21:13:05.331666 16528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance:
uuid: "fcb0892c755540c68dc6216f2b5999ec"
format_stamp: "Formatted at 2026-09-24 21:13:05 on dist-test-slave-q276"
server_key: "4e16e65ffe110cdd43d700e04eab6b10"
server_key_iv: "a6947b6bd7f092db79b483b68aabafb8"
server_key_version: "encryptionkey@0"
I20260924 21:13:05.332990 16528 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:13:05.333735 16542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:05.333945 16528 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:13:05.334039 16528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/wal
uuid: "fcb0892c755540c68dc6216f2b5999ec"
format_stamp: "Formatted at 2026-09-24 21:13:05 on dist-test-slave-q276"
server_key: "4e16e65ffe110cdd43d700e04eab6b10"
server_key_iv: "a6947b6bd7f092db79b483b68aabafb8"
server_key_version: "encryptionkey@0"
I20260924 21:13:05.334124 16528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:13:05.344795 16528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:13:05.345077 16528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:13:05.345217 16528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:13:05.345425 16528 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:13:05.345736 16528 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:13:05.345793 16528 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:05.345835 16528 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:13:05.345861 16528 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:05.351621 16528 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.131:32769
I20260924 21:13:05.351670 16655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.131:32769 every 8 connection(s)
I20260924 21:13:05.351977 16528 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
I20260924 21:13:05.353163 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 16528
I20260924 21:13:05.353273 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance
I20260924 21:13:05.353483 6986 external_mini_cluster.cc:1428] Setting key 643ccc75d43b26f769fd2aca6481413a
I20260924 21:13:05.357630 16245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:05.352675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:49625 (local address 127.6.210.190:39271)
0924 21:13:05.352745 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:05.352748 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:05.352989 (+ 241us) server_negotiation.cc:407] Connection header received
0924 21:13:05.353857 (+ 868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:05.353858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:05.353870 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:05.353921 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:05.354810 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:05.355271 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:05.356333 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:05.356504 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:05.356553 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:05.357408 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:13:05.357409 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:13:05.357409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:13:05.357413 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:13:05.357425 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:13:05.357429 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:13:05.357471 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0924 21:13:05.357557 (+ 86us) server_negotiation.cc:299] Negotiation successful
0924 21:13:05.357576 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 21:13:05.357899 16658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:05.352661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39271 (local address 127.6.210.131:49625)
0924 21:13:05.352865 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:05.352900 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:05.353739 (+ 839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:05.353995 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:05.353999 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:05.354010 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:13:05.354731 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:05.354737 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:05.355361 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:05.355362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:05.356238 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:05.356240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:05.356581 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:05.356581 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:05.356704 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:05.357277 (+ 573us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:13:05.357278 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:13:05.357308 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:13:05.357330 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:13:05.357487 (+ 157us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:13:05.357491 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:13:05.357493 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 21:13:05.357548 (+ 55us) client_negotiation.cc:240] Negotiation successful
0924 21:13:05.357629 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":103,"threads_started":1}
I20260924 21:13:05.358491 16656 heartbeater.cc:347] Connected to a master server at 127.6.210.190:39271
I20260924 21:13:05.358596 16656 heartbeater.cc:464] Registering TS with master...
I20260924 21:13:05.358861 16656 heartbeater.cc:511] Master 127.6.210.190:39271 requested a full tablet report, sending...
I20260924 21:13:05.359354 16206 ts_manager.cc:194] Registered new tserver with Master: fcb0892c755540c68dc6216f2b5999ec (127.6.210.131:32769)
I20260924 21:13:05.359786 16206 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.210.131:49625
I20260924 21:13:05.365480 6986 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260924 21:13:05.367576 16245 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:13:05.367055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54460 (local address 127.6.210.190:39271)
0924 21:13:05.367122 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:05.367125 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:05.367237 (+ 112us) server_negotiation.cc:407] Connection header received
0924 21:13:05.367298 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:05.367299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:05.367307 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:05.367351 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:05.367497 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54460: BlockingRecv error: recv got EOF from 127.0.0.1:54460 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 21:13:05.367622 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 16266
I20260924 21:13:05.372421 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 16397
I20260924 21:13:05.376797 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 16528
I20260924 21:13:05.381295 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 16175
2026-09-24T21:13:05Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (623 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-24T21:13:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:13:05Z Disabled control of system clock
I20260924 21:13:05.408291 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:44865
--webserver_interface=127.6.210.190
--webserver_port=0
--builtin_ntp_servers=127.6.210.148:39315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:44865
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260924 21:13:05.492115 16667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:13:05.492341 16667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:13:05.492363 16667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:13:05.493494 16667 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:13:05.493532 16667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:13:05.493544 16667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:13:05.493557 16667 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:13:05.493569 16667 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:13:05.494925 16667 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:39315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:44865
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:44865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:13:05.495127 16667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:13:05.495357 16667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 21:13:05.500684 16667 server_base.cc:1034] running on GCE node
W20260924 21:13:05.500639 16675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:13:05.500629 16673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:13:05.500620 16672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:13:05.501154 16667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:13:05.501494 16667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:13:05.502656 16667 hybrid_clock.cc:648] HybridClock initialized: now 1790284385502631 us; error 60 us; skew 500 ppm
I20260924 21:13:05.503890 16667 webserver.cc:460] Webserver started at http://127.6.210.190:46177/ using document root <none> and password file <none>
I20260924 21:13:05.504119 16667 fs_manager.cc:359] Metadata directory not provided
I20260924 21:13:05.504186 16667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:13:05.504364 16667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:13:05.505431 16667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "632efc814c0047b5b1a82539a08331b3"
format_stamp: "Formatted at 2026-09-24 21:13:05 on dist-test-slave-q276"
server_key: "4f0106672098e2ee14a294ca78b18ebe"
server_key_iv: "b141452b2bd3512528b9b2731e77c194"
server_key_version: "encryptionkey@0"
I20260924 21:13:05.505796 16667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "632efc814c0047b5b1a82539a08331b3"
format_stamp: "Formatted at 2026-09-24 21:13:05 on dist-test-slave-q276"
server_key: "4f0106672098e2ee14a294ca78b18ebe"
server_key_iv: "b141452b2bd3512528b9b2731e77c194"
server_key_version: "encryptionkey@0"
I20260924 21:13:05.507153 16667 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:13:05.507896 16681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:05.508065 16667 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:13:05.508160 16667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "632efc814c0047b5b1a82539a08331b3"
format_stamp: "Formatted at 2026-09-24 21:13:05 on dist-test-slave-q276"
server_key: "4f0106672098e2ee14a294ca78b18ebe"
server_key_iv: "b141452b2bd3512528b9b2731e77c194"
server_key_version: "encryptionkey@0"
I20260924 21:13:05.508237 16667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:13:05.511729 16667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:13:05.512003 16667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:13:05.512149 16667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:13:05.515930 16667 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:44865
I20260924 21:13:05.515959 16733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:44865 every 8 connection(s)
I20260924 21:13:05.516294 16667 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:13:05.516965 16734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:13:05.519788 16734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 632efc814c0047b5b1a82539a08331b3: Bootstrap starting.
I20260924 21:13:05.520309 16734 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 632efc814c0047b5b1a82539a08331b3: Neither blocks nor log segments found. Creating new log.
I20260924 21:13:05.520539 16734 log.cc:824] T 00000000000000000000000000000000 P 632efc814c0047b5b1a82539a08331b3: Log is configured to *not* fsync() on all Append() calls
I20260924 21:13:05.521194 16734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 632efc814c0047b5b1a82539a08331b3: No bootstrap required, opened a new log
I20260924 21:13:05.522677 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 16667
I20260924 21:13:05.522770 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:13:05.522943 6986 external_mini_cluster.cc:1428] Setting key 652b2c4d0ab2c8c43e88bee0529ba494
I20260924 21:13:05.523015 16734 raft_consensus.cc:348] T 00000000000000000000000000000000 P 632efc814c0047b5b1a82539a08331b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "632efc814c0047b5b1a82539a08331b3" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 44865 } }
I20260924 21:13:05.523195 16734 raft_consensus.cc:374] T 00000000000000000000000000000000 P 632efc814c0047b5b1a82539a08331b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:13:05.523222 16734 raft_consensus.cc:729] T 00000000000000000000000000000000 P 632efc814c0047b5b1a82539a08331b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632efc814c0047b5b1a82539a08331b3, State: Initialized, Role: FOLLOWER
I20260924 21:13:05.523384 16734 consensus_queue.cc:260] T 00000000000000000000000000000000 P 632efc814c0047b5b1a82539a08331b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "632efc814c0047b5b1a82539a08331b3" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 44865 } }
I20260924 21:13:05.523487 16734 raft_consensus.cc:388] T 00000000000000000000000000000000 P 632efc814c0047b5b1a82539a08331b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:13:05.523530 16734 raft_consensus.cc:482] T 00000000000000000000000000000000 P 632efc814c0047b5b1a82539a08331b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:13:05.523569 16734 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 632efc814c0047b5b1a82539a08331b3 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:13:05.524087 16734 raft_consensus.cc:504] T 00000000000000000000000000000000 P 632efc814c0047b5b1a82539a08331b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "632efc814c0047b5b1a82539a08331b3" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 44865 } }
I20260924 21:13:05.524212 16734 leader_election.cc:302] T 00000000000000000000000000000000 P 632efc814c0047b5b1a82539a08331b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 632efc814c0047b5b1a82539a08331b3; no voters:
I20260924 21:13:05.524672 16734 leader_election.cc:288] T 00000000000000000000000000000000 P 632efc814c0047b5b1a82539a08331b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:13:05.524718 16739 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 632efc814c0047b5b1a82539a08331b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:13:05.524880 16739 raft_consensus.cc:686] T 00000000000000000000000000000000 P 632efc814c0047b5b1a82539a08331b3 [term 1 LEADER]: Becoming Leader. State: Replica: 632efc814c0047b5b1a82539a08331b3, State: Running, Role: LEADER
I20260924 21:13:05.524942 16734 sys_catalog.cc:564] T 00000000000000000000000000000000 P 632efc814c0047b5b1a82539a08331b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:13:05.524966 16739 consensus_queue.cc:237] T 00000000000000000000000000000000 P 632efc814c0047b5b1a82539a08331b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "632efc814c0047b5b1a82539a08331b3" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 44865 } }
I20260924 21:13:05.525298 16740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 632efc814c0047b5b1a82539a08331b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "632efc814c0047b5b1a82539a08331b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "632efc814c0047b5b1a82539a08331b3" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 44865 } } }
I20260924 21:13:05.525388 16740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 632efc814c0047b5b1a82539a08331b3 [sys.catalog]: This master's current role is: LEADER
I20260924 21:13:05.525290 16739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 632efc814c0047b5b1a82539a08331b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 632efc814c0047b5b1a82539a08331b3. Latest consensus state: current_term: 1 leader_uuid: "632efc814c0047b5b1a82539a08331b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "632efc814c0047b5b1a82539a08331b3" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 44865 } } }
I20260924 21:13:05.525538 16739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 632efc814c0047b5b1a82539a08331b3 [sys.catalog]: This master's current role is: LEADER
I20260924 21:13:05.525725 16738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:05.523536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44932 (local address 127.6.210.190:44865)
0924 21:13:05.523788 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:05.523804 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:05.523831 (+ 27us) server_negotiation.cc:407] Connection header received
0924 21:13:05.523926 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:05.523931 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:05.523963 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:05.524097 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:05.524722 (+ 625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:13:05.524726 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:13:05.524727 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:13:05.524732 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:13:05.524783 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:13:05.524799 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:13:05.525021 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0924 21:13:05.525053 (+ 32us) server_negotiation.cc:299] Negotiation successful
0924 21:13:05.525115 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":93,"threads_started":1}
I20260924 21:13:05.526328 16745 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:13:05.527015 16745 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:13:05.528381 16745 catalog_manager.cc:1347] Generated new cluster ID: 13d8c4adb6aa4caaaa595c287ff400ad
I20260924 21:13:05.528431 16745 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:13:05.533471 16745 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:13:05.533633 16745 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:13:05.536769 16745 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 632efc814c0047b5b1a82539a08331b3: Generated new TSK 0
I20260924 21:13:05.536959 16745 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:13:05.542162 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.129:0
--local_ip_for_outbound_sockets=127.6.210.129
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:44865
--builtin_ntp_servers=127.6.210.148:39315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260924 21:13:05.624171 16758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:13:05.624486 16758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:13:05.624526 16758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:13:05.625885 16758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:13:05.625923 16758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:13:05.625950 16758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.129
I20260924 21:13:05.627478 16758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:39315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:44865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.129
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:13:05.627700 16758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:13:05.628000 16758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:13:05.633994 16766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:13:05.633988 16764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:13:05.633996 16763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:13:05.634377 16758 server_base.cc:1034] running on GCE node
I20260924 21:13:05.634614 16758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:13:05.634927 16758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:13:05.636101 16758 hybrid_clock.cc:648] HybridClock initialized: now 1790284385636084 us; error 31 us; skew 500 ppm
I20260924 21:13:05.637516 16758 webserver.cc:460] Webserver started at http://127.6.210.129:46591/ using document root <none> and password file <none>
I20260924 21:13:05.637750 16758 fs_manager.cc:359] Metadata directory not provided
I20260924 21:13:05.637830 16758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:13:05.637984 16758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:13:05.639124 16758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/data/instance:
uuid: "9b63c9c938664ed99201abcaa2ea5eb1"
format_stamp: "Formatted at 2026-09-24 21:13:05 on dist-test-slave-q276"
server_key: "51c79b42b461c16001609a3c7992ad88"
server_key_iv: "67856d0346f6369bb3b4aa0c1abfb19a"
server_key_version: "encryptionkey@0"
I20260924 21:13:05.639489 16758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance:
uuid: "9b63c9c938664ed99201abcaa2ea5eb1"
format_stamp: "Formatted at 2026-09-24 21:13:05 on dist-test-slave-q276"
server_key: "51c79b42b461c16001609a3c7992ad88"
server_key_iv: "67856d0346f6369bb3b4aa0c1abfb19a"
server_key_version: "encryptionkey@0"
I20260924 21:13:05.640834 16758 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 21:13:05.641604 16772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:05.641804 16758 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:13:05.641901 16758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/wal
uuid: "9b63c9c938664ed99201abcaa2ea5eb1"
format_stamp: "Formatted at 2026-09-24 21:13:05 on dist-test-slave-q276"
server_key: "51c79b42b461c16001609a3c7992ad88"
server_key_iv: "67856d0346f6369bb3b4aa0c1abfb19a"
server_key_version: "encryptionkey@0"
I20260924 21:13:05.641971 16758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:13:05.645320 16758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:13:05.645592 16758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:13:05.645733 16758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:13:05.645942 16758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:13:05.646242 16758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:13:05.646322 16758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:05.646360 16758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:13:05.646385 16758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:05.651799 16758 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.129:34765
I20260924 21:13:05.651880 16885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.129:34765 every 8 connection(s)
I20260924 21:13:05.652192 16758 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
I20260924 21:13:05.655580 16738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:05.652950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:38573 (local address 127.6.210.190:44865)
0924 21:13:05.653013 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:05.653015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:05.653256 (+ 241us) server_negotiation.cc:407] Connection header received
0924 21:13:05.653825 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:05.653826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:05.653834 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:05.653876 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:05.655165 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:13:05.655166 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:13:05.655167 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:13:05.655171 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:13:05.655186 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:13:05.655190 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:13:05.655363 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0924 21:13:05.655486 (+ 123us) server_negotiation.cc:299] Negotiation successful
0924 21:13:05.655505 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 21:13:05.655828 16888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:05.652946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:44865 (local address 127.6.210.129:38573)
0924 21:13:05.653112 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:05.653158 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:05.653706 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:05.653967 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:05.653970 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:05.653979 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:13:05.654991 (+ 1012us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:13:05.654992 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:13:05.655020 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:13:05.655048 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:13:05.655360 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:13:05.655369 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:13:05.655372 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:13:05.655493 (+ 121us) client_negotiation.cc:240] Negotiation successful
0924 21:13:05.655561 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":69,"threads_started":1}
I20260924 21:13:05.656437 16886 heartbeater.cc:347] Connected to a master server at 127.6.210.190:44865
I20260924 21:13:05.656543 16886 heartbeater.cc:464] Registering TS with master...
I20260924 21:13:05.656713 16886 heartbeater.cc:511] Master 127.6.210.190:44865 requested a full tablet report, sending...
I20260924 21:13:05.657323 16698 ts_manager.cc:194] Registered new tserver with Master: 9b63c9c938664ed99201abcaa2ea5eb1 (127.6.210.129:34765)
I20260924 21:13:05.657735 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 16758
I20260924 21:13:05.657841 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance
I20260924 21:13:05.658035 6986 external_mini_cluster.cc:1428] Setting key 7bedb1689e4beb4a2b4ab01653b887a2
I20260924 21:13:05.658965 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.130:0
--local_ip_for_outbound_sockets=127.6.210.130
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:44865
--builtin_ntp_servers=127.6.210.148:39315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260924 21:13:05.738986 16889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:13:05.739234 16889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:13:05.739270 16889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:13:05.740396 16889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:13:05.740463 16889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:13:05.740506 16889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.130
I20260924 21:13:05.741816 16889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:39315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:44865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.130
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:13:05.742085 16889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:13:05.742390 16889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:13:05.747515 16894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:13:05.747599 16889 server_base.cc:1034] running on GCE node
W20260924 21:13:05.747511 16895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:13:05.747581 16897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:13:05.747991 16889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:13:05.748243 16889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:13:05.749419 16889 hybrid_clock.cc:648] HybridClock initialized: now 1790284385749417 us; error 35 us; skew 500 ppm
I20260924 21:13:05.750731 16889 webserver.cc:460] Webserver started at http://127.6.210.130:36681/ using document root <none> and password file <none>
I20260924 21:13:05.750995 16889 fs_manager.cc:359] Metadata directory not provided
I20260924 21:13:05.751065 16889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:13:05.751272 16889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:13:05.752390 16889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/data/instance:
uuid: "a0679daf4d0f45dfa5a149a30a4cce93"
format_stamp: "Formatted at 2026-09-24 21:13:05 on dist-test-slave-q276"
server_key: "c8ab2fef97acf2fe07b7671e60bb448b"
server_key_iv: "5f3d9a6ef02760444ba2ef6d07057c96"
server_key_version: "encryptionkey@0"
I20260924 21:13:05.752756 16889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance:
uuid: "a0679daf4d0f45dfa5a149a30a4cce93"
format_stamp: "Formatted at 2026-09-24 21:13:05 on dist-test-slave-q276"
server_key: "c8ab2fef97acf2fe07b7671e60bb448b"
server_key_iv: "5f3d9a6ef02760444ba2ef6d07057c96"
server_key_version: "encryptionkey@0"
I20260924 21:13:05.754118 16889 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:13:05.754913 16903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:05.755071 16889 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:13:05.755146 16889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/wal
uuid: "a0679daf4d0f45dfa5a149a30a4cce93"
format_stamp: "Formatted at 2026-09-24 21:13:05 on dist-test-slave-q276"
server_key: "c8ab2fef97acf2fe07b7671e60bb448b"
server_key_iv: "5f3d9a6ef02760444ba2ef6d07057c96"
server_key_version: "encryptionkey@0"
I20260924 21:13:05.755241 16889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:13:05.758574 16889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:13:05.758809 16889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:13:05.758903 16889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:13:05.759055 16889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:13:05.759299 16889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:13:05.759328 16889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:05.759348 16889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:13:05.759361 16889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:05.765338 16889 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.130:38951
I20260924 21:13:05.765416 17016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.130:38951 every 8 connection(s)
I20260924 21:13:05.765694 16889 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
I20260924 21:13:05.768626 16738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:05.766459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:59057 (local address 127.6.210.190:44865)
0924 21:13:05.766525 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:05.766527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:05.766760 (+ 233us) server_negotiation.cc:407] Connection header received
0924 21:13:05.767204 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:05.767205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:05.767211 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:05.767287 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:05.768433 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:13:05.768434 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:13:05.768434 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:13:05.768437 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:13:05.768450 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:13:05.768453 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:13:05.768505 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 21:13:05.768564 (+ 59us) server_negotiation.cc:299] Negotiation successful
0924 21:13:05.768576 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 21:13:05.768796 17019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:05.766444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:44865 (local address 127.6.210.130:59057)
0924 21:13:05.766626 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:05.766665 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:05.767113 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:05.767339 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:05.767342 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:05.767350 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:13:05.768302 (+ 952us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:13:05.768302 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:13:05.768330 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:13:05.768357 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:13:05.768505 (+ 148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:13:05.768508 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:13:05.768510 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 21:13:05.768560 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 21:13:05.768604 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":92,"threads_started":1}
I20260924 21:13:05.769269 17017 heartbeater.cc:347] Connected to a master server at 127.6.210.190:44865
I20260924 21:13:05.769346 17017 heartbeater.cc:464] Registering TS with master...
I20260924 21:13:05.769491 17017 heartbeater.cc:511] Master 127.6.210.190:44865 requested a full tablet report, sending...
I20260924 21:13:05.769860 16698 ts_manager.cc:194] Registered new tserver with Master: a0679daf4d0f45dfa5a149a30a4cce93 (127.6.210.130:38951)
I20260924 21:13:05.773447 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 16889
I20260924 21:13:05.773576 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance
I20260924 21:13:05.773787 6986 external_mini_cluster.cc:1428] Setting key e28105c5bd86d8d42d9d4d344a916ea1
I20260924 21:13:05.774798 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.131:0
--local_ip_for_outbound_sockets=127.6.210.131
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:44865
--builtin_ntp_servers=127.6.210.148:39315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260924 21:13:05.854984 17020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:13:05.855178 17020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:13:05.855199 17020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:13:05.856299 17020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:13:05.856334 17020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:13:05.856359 17020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.131
I20260924 21:13:05.857647 17020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:39315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:44865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.131
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:13:05.857853 17020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:13:05.858086 17020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:13:05.863139 17026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:13:05.863200 17020 server_base.cc:1034] running on GCE node
W20260924 21:13:05.863256 17025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:13:05.863150 17028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:13:05.863508 17020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:13:05.863726 17020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:13:05.864885 17020 hybrid_clock.cc:648] HybridClock initialized: now 1790284385864871 us; error 26 us; skew 500 ppm
I20260924 21:13:05.866194 17020 webserver.cc:460] Webserver started at http://127.6.210.131:42341/ using document root <none> and password file <none>
I20260924 21:13:05.866456 17020 fs_manager.cc:359] Metadata directory not provided
I20260924 21:13:05.866503 17020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:13:05.866663 17020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:13:05.867741 17020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/data/instance:
uuid: "d906b6af932c40a6983856b01ec72af5"
format_stamp: "Formatted at 2026-09-24 21:13:05 on dist-test-slave-q276"
server_key: "0a32a9187ebecf087c1982669f8cab1d"
server_key_iv: "a25ec17e79684a2b636ba065174a61d4"
server_key_version: "encryptionkey@0"
I20260924 21:13:05.868111 17020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance:
uuid: "d906b6af932c40a6983856b01ec72af5"
format_stamp: "Formatted at 2026-09-24 21:13:05 on dist-test-slave-q276"
server_key: "0a32a9187ebecf087c1982669f8cab1d"
server_key_iv: "a25ec17e79684a2b636ba065174a61d4"
server_key_version: "encryptionkey@0"
I20260924 21:13:05.869426 17020 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:13:05.870153 17034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:05.870370 17020 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:13:05.870440 17020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/wal
uuid: "d906b6af932c40a6983856b01ec72af5"
format_stamp: "Formatted at 2026-09-24 21:13:05 on dist-test-slave-q276"
server_key: "0a32a9187ebecf087c1982669f8cab1d"
server_key_iv: "a25ec17e79684a2b636ba065174a61d4"
server_key_version: "encryptionkey@0"
I20260924 21:13:05.870530 17020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:13:05.873785 17020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:13:05.874039 17020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:13:05.874178 17020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:13:05.874395 17020 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:13:05.874707 17020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:13:05.874740 17020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:05.874794 17020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:13:05.874821 17020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:05.880293 17020 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.131:39331
I20260924 21:13:05.880333 17147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.131:39331 every 8 connection(s)
I20260924 21:13:05.880676 17020 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
I20260924 21:13:05.883684 16738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:05.881402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:43325 (local address 127.6.210.190:44865)
0924 21:13:05.881460 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:05.881462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:05.881707 (+ 245us) server_negotiation.cc:407] Connection header received
0924 21:13:05.882258 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:05.882275 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:05.882282 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:05.882338 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:05.883422 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:13:05.883423 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:13:05.883424 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:13:05.883428 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:13:05.883444 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:13:05.883448 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:13:05.883502 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 21:13:05.883597 (+ 95us) server_negotiation.cc:299] Negotiation successful
0924 21:13:05.883612 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 21:13:05.883913 17150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:05.881387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:44865 (local address 127.6.210.131:43325)
0924 21:13:05.881589 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:05.881625 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:05.882090 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:05.882412 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:05.882416 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:05.882423 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:13:05.883269 (+ 846us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:13:05.883270 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:13:05.883296 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:13:05.883323 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:13:05.883508 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:13:05.883512 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:13:05.883516 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:13:05.883608 (+ 92us) client_negotiation.cc:240] Negotiation successful
0924 21:13:05.883687 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":88,"threads_started":1}
I20260924 21:13:05.884378 17148 heartbeater.cc:347] Connected to a master server at 127.6.210.190:44865
I20260924 21:13:05.884449 17148 heartbeater.cc:464] Registering TS with master...
I20260924 21:13:05.884547 17148 heartbeater.cc:511] Master 127.6.210.190:44865 requested a full tablet report, sending...
I20260924 21:13:05.884860 16698 ts_manager.cc:194] Registered new tserver with Master: d906b6af932c40a6983856b01ec72af5 (127.6.210.131:39331)
I20260924 21:13:05.889299 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 17020
I20260924 21:13:05.889375 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance
I20260924 21:13:05.889557 6986 external_mini_cluster.cc:1428] Setting key 201883325494e5225633a84cb5a68137
I20260924 21:13:05.890547 6986 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260924 21:13:05.892261 16738 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:13:05.891807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44942 (local address 127.6.210.190:44865)
0924 21:13:05.891850 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:05.891852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:05.891968 (+ 116us) server_negotiation.cc:407] Connection header received
0924 21:13:05.892024 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:05.892025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:05.892029 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:05.892080 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:05.892198 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44942: BlockingRecv error: recv got EOF from 127.0.0.1:44942 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 21:13:05.892370 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 16758
I20260924 21:13:05.896590 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 16889
I20260924 21:13:05.900909 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 17020
I20260924 21:13:05.905242 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 16667
2026-09-24T21:13:05Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (523 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:13:05 dist-test-slave-q276 krb5kdc[17158](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:13:05 dist-test-slave-q276 krb5kdc[17158](info): set up 2 sockets
Sep 24 21:13:05 dist-test-slave-q276 krb5kdc[17158](info): commencing operation
krb5kdc: starting...
W20260924 21:13:06.943244 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:13:07 dist-test-slave-q276 krb5kdc[17158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:13:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:13:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.210.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
I20260924 21:13:07.064987 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:33035
--webserver_interface=127.6.210.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:39951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:33035
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:13:07.144553 17171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:13:07.144832 17171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:13:07.144877 17171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:13:07.146106 17171 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:13:07.146183 17171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:13:07.146219 17171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:13:07.146252 17171 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:13:07.146315 17171 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:13:07.147866 17171 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:39951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:33035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:33035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:13:07.148146 17171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:13:07.148384 17171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:13:07.153445 17176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:13:07.153514 17171 server_base.cc:1034] running on GCE node
W20260924 21:13:07.153437 17177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:13:07.153450 17179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:13:07.153981 17171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:13:07.154245 17171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:13:07.155501 17171 hybrid_clock.cc:648] HybridClock initialized: now 1790284387155497 us; error 58 us; skew 500 ppm
Sep 24 21:13:07 dist-test-slave-q276 krb5kdc[17158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284387, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:13:07.158238 17171 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:13:07.158825 17171 webserver.cc:460] Webserver started at http://127.6.210.190:42193/ using document root <none> and password file <none>
I20260924 21:13:07.159076 17171 fs_manager.cc:359] Metadata directory not provided
I20260924 21:13:07.159147 17171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:13:07.159294 17171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:13:07.160360 17171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "550f4b53a02248ae98aeec768aafa75b"
format_stamp: "Formatted at 2026-09-24 21:13:07 on dist-test-slave-q276"
server_key: "f4e0fd523a21cf1a39baeea87f6c07c3"
server_key_iv: "c91d7b6a09575490e4016d9b4c248d34"
server_key_version: "encryptionkey@0"
I20260924 21:13:07.160727 17171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "550f4b53a02248ae98aeec768aafa75b"
format_stamp: "Formatted at 2026-09-24 21:13:07 on dist-test-slave-q276"
server_key: "f4e0fd523a21cf1a39baeea87f6c07c3"
server_key_iv: "c91d7b6a09575490e4016d9b4c248d34"
server_key_version: "encryptionkey@0"
I20260924 21:13:07.162045 17171 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:13:07.162894 17186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:07.163127 17171 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 21:13:07.163203 17171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "550f4b53a02248ae98aeec768aafa75b"
format_stamp: "Formatted at 2026-09-24 21:13:07 on dist-test-slave-q276"
server_key: "f4e0fd523a21cf1a39baeea87f6c07c3"
server_key_iv: "c91d7b6a09575490e4016d9b4c248d34"
server_key_version: "encryptionkey@0"
I20260924 21:13:07.163254 17171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:13:07.176054 17171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:13:07.178383 17171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:13:07.178499 17171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:13:07.182348 17171 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:33035
I20260924 21:13:07.182374 17238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:33035 every 8 connection(s)
I20260924 21:13:07.182715 17171 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:13:07.183624 17239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:13:07.185879 17239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 550f4b53a02248ae98aeec768aafa75b: Bootstrap starting.
I20260924 21:13:07.186511 17239 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 550f4b53a02248ae98aeec768aafa75b: Neither blocks nor log segments found. Creating new log.
I20260924 21:13:07.186792 17239 log.cc:824] T 00000000000000000000000000000000 P 550f4b53a02248ae98aeec768aafa75b: Log is configured to *not* fsync() on all Append() calls
I20260924 21:13:07.187465 17239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 550f4b53a02248ae98aeec768aafa75b: No bootstrap required, opened a new log
I20260924 21:13:07.189441 17239 raft_consensus.cc:348] T 00000000000000000000000000000000 P 550f4b53a02248ae98aeec768aafa75b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "550f4b53a02248ae98aeec768aafa75b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 33035 } }
I20260924 21:13:07.189662 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 17171
I20260924 21:13:07.189669 17239 raft_consensus.cc:374] T 00000000000000000000000000000000 P 550f4b53a02248ae98aeec768aafa75b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:13:07.189718 17239 raft_consensus.cc:729] T 00000000000000000000000000000000 P 550f4b53a02248ae98aeec768aafa75b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 550f4b53a02248ae98aeec768aafa75b, State: Initialized, Role: FOLLOWER
I20260924 21:13:07.189756 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:13:07.189854 17239 consensus_queue.cc:260] T 00000000000000000000000000000000 P 550f4b53a02248ae98aeec768aafa75b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "550f4b53a02248ae98aeec768aafa75b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 33035 } }
I20260924 21:13:07.189931 17239 raft_consensus.cc:388] T 00000000000000000000000000000000 P 550f4b53a02248ae98aeec768aafa75b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:13:07.189961 17239 raft_consensus.cc:482] T 00000000000000000000000000000000 P 550f4b53a02248ae98aeec768aafa75b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:13:07.189989 6986 external_mini_cluster.cc:1428] Setting key decad778100be5301390c48255462de9
I20260924 21:13:07.190008 17239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 550f4b53a02248ae98aeec768aafa75b [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:13:07.190749 17239 raft_consensus.cc:504] T 00000000000000000000000000000000 P 550f4b53a02248ae98aeec768aafa75b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "550f4b53a02248ae98aeec768aafa75b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 33035 } }
I20260924 21:13:07.190862 17239 leader_election.cc:302] T 00000000000000000000000000000000 P 550f4b53a02248ae98aeec768aafa75b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 550f4b53a02248ae98aeec768aafa75b; no voters:
I20260924 21:13:07.191022 17239 leader_election.cc:288] T 00000000000000000000000000000000 P 550f4b53a02248ae98aeec768aafa75b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:13:07.191044 17244 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 550f4b53a02248ae98aeec768aafa75b [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:13:07.191202 17244 raft_consensus.cc:686] T 00000000000000000000000000000000 P 550f4b53a02248ae98aeec768aafa75b [term 1 LEADER]: Becoming Leader. State: Replica: 550f4b53a02248ae98aeec768aafa75b, State: Running, Role: LEADER
I20260924 21:13:07.191273 17239 sys_catalog.cc:564] T 00000000000000000000000000000000 P 550f4b53a02248ae98aeec768aafa75b [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:13:07.191416 17244 consensus_queue.cc:237] T 00000000000000000000000000000000 P 550f4b53a02248ae98aeec768aafa75b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "550f4b53a02248ae98aeec768aafa75b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 33035 } }
I20260924 21:13:07.191753 17245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 550f4b53a02248ae98aeec768aafa75b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 550f4b53a02248ae98aeec768aafa75b. Latest consensus state: current_term: 1 leader_uuid: "550f4b53a02248ae98aeec768aafa75b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "550f4b53a02248ae98aeec768aafa75b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 33035 } } }
I20260924 21:13:07.191849 17245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 550f4b53a02248ae98aeec768aafa75b [sys.catalog]: This master's current role is: LEADER
I20260924 21:13:07.191991 17246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 550f4b53a02248ae98aeec768aafa75b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "550f4b53a02248ae98aeec768aafa75b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "550f4b53a02248ae98aeec768aafa75b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 33035 } } }
I20260924 21:13:07.192060 17246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 550f4b53a02248ae98aeec768aafa75b [sys.catalog]: This master's current role is: LEADER
I20260924 21:13:07.192628 17250 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:13:07.193086 17250 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:13:07.194406 17250 catalog_manager.cc:1347] Generated new cluster ID: 4e668e782ff14557a8e908bc6b4e7e18
I20260924 21:13:07.194455 17250 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 21:13:07 dist-test-slave-q276 krb5kdc[17158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:13:07.201699 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:07.190465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43536 (local address 127.6.210.190:33035)
0924 21:13:07.191049 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:07.191060 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:07.191096 (+ 36us) server_negotiation.cc:407] Connection header received
0924 21:13:07.191211 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:07.191217 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:07.191272 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:07.192181 (+ 909us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:07.192273 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.193553 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.194638 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.195214 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.195310 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:07.198076 (+ 2766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:13:07.198081 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:13:07.198081 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:13:07.198093 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:13:07.200113 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:13:07.200490 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:13:07.200493 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:13:07.200494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:13:07.200610 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:13:07.200810 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:13:07.200811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:13:07.200812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:13:07.201121 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:13:07.201288 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0924 21:13:07.201360 (+ 72us) server_negotiation.cc:299] Negotiation successful
0924 21:13:07.201497 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":530,"thread_start_us":499,"threads_started":1}
I20260924 21:13:07.203226 17250 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:13:07.203934 17250 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:13:07.210431 17250 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 550f4b53a02248ae98aeec768aafa75b: Generated new TSK 0
I20260924 21:13:07.210597 17250 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.210.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
I20260924 21:13:07.241236 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.129:0
--local_ip_for_outbound_sockets=127.6.210.129
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:33035
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:39951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:13:07.322741 17265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:13:07.322943 17265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:13:07.322968 17265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:13:07.324119 17265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:13:07.324169 17265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:13:07.324194 17265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.129
I20260924 21:13:07.325716 17265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:39951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:33035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.129
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:13:07.325940 17265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:13:07.326189 17265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:13:07.331152 17270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:13:07.331205 17271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:13:07.331244 17273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:13:07.331292 17265 server_base.cc:1034] running on GCE node
I20260924 21:13:07.331503 17265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:13:07.331756 17265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:13:07.332927 17265 hybrid_clock.cc:648] HybridClock initialized: now 1790284387332922 us; error 36 us; skew 500 ppm
Sep 24 21:13:07 dist-test-slave-q276 krb5kdc[17158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284387, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:13:07.335369 17265 init.cc:377] Logged in from keytab as kudu/127.6.210.129@KRBTEST.COM (short username kudu)
I20260924 21:13:07.335865 17265 webserver.cc:460] Webserver started at http://127.6.210.129:34327/ using document root <none> and password file <none>
I20260924 21:13:07.336083 17265 fs_manager.cc:359] Metadata directory not provided
I20260924 21:13:07.336154 17265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:13:07.336302 17265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:13:07.337342 17265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/data/instance:
uuid: "9acde0d66caa449499a2d4f1debc14e8"
format_stamp: "Formatted at 2026-09-24 21:13:07 on dist-test-slave-q276"
server_key: "3143a6d60a6b3e18c5d3121a4f819bb7"
server_key_iv: "973ce2bb2ddf2302fa55fb36de6ea4f3"
server_key_version: "encryptionkey@0"
I20260924 21:13:07.337702 17265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance:
uuid: "9acde0d66caa449499a2d4f1debc14e8"
format_stamp: "Formatted at 2026-09-24 21:13:07 on dist-test-slave-q276"
server_key: "3143a6d60a6b3e18c5d3121a4f819bb7"
server_key_iv: "973ce2bb2ddf2302fa55fb36de6ea4f3"
server_key_version: "encryptionkey@0"
I20260924 21:13:07.339133 17265 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:13:07.339972 17280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:07.340140 17265 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:13:07.340291 17265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/wal
uuid: "9acde0d66caa449499a2d4f1debc14e8"
format_stamp: "Formatted at 2026-09-24 21:13:07 on dist-test-slave-q276"
server_key: "3143a6d60a6b3e18c5d3121a4f819bb7"
server_key_iv: "973ce2bb2ddf2302fa55fb36de6ea4f3"
server_key_version: "encryptionkey@0"
I20260924 21:13:07.340370 17265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:13:07.355510 17265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:13:07.357519 17265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:13:07.357683 17265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:13:07.357924 17265 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:13:07.358232 17265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:13:07.358309 17265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:07.358356 17265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:13:07.358379 17265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:07.364389 17265 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.129:42681
I20260924 21:13:07.364430 17393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.129:42681 every 8 connection(s)
I20260924 21:13:07.364745 17265 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
I20260924 21:13:07.366019 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 17265
I20260924 21:13:07.366128 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance
I20260924 21:13:07.366351 6986 external_mini_cluster.cc:1428] Setting key 1b698cfc20411432eff9383065abb19d
Sep 24 21:13:07 dist-test-slave-q276 krb5kdc[17158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284387, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.210.130@KRBTEST.COM; defaulting to no policy
I20260924 21:13:07.375959 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:07.365449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:45985 (local address 127.6.210.190:33035)
0924 21:13:07.365534 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:07.365536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:07.365757 (+ 221us) server_negotiation.cc:407] Connection header received
0924 21:13:07.366409 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:07.366411 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:07.366426 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:07.366502 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:07.367652 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.368483 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.370015 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.370205 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.370374 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:07.372815 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:13:07.372816 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:13:07.372817 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:13:07.372825 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:13:07.374523 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:13:07.374986 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:13:07.374988 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:13:07.374988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:13:07.375049 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:13:07.375317 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:13:07.375318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:13:07.375319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:13:07.375472 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:13:07.375540 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 21:13:07.375822 (+ 282us) server_negotiation.cc:299] Negotiation successful
0924 21:13:07.375890 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260924 21:13:07.376240 17396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:07.365425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:33035 (local address 127.6.210.129:45985)
0924 21:13:07.365610 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:07.365662 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:07.366180 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:07.366607 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:07.366612 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:07.367359 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:13:07.367554 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:07.367558 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.368589 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.368590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:07.369930 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:07.369933 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.370377 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.370378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:07.370542 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:07.371121 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:13:07.371132 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:13:07.372717 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:13:07.374615 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:13:07.374621 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:13:07.374623 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:13:07.374896 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:13:07.375127 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:13:07.375127 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:13:07.375128 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:13:07.375240 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:13:07.375550 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:13:07.375555 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:13:07.375691 (+ 136us) client_negotiation.cc:769] Sending connection context
0924 21:13:07.375784 (+ 93us) client_negotiation.cc:240] Negotiation successful
0924 21:13:07.375915 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":103,"threads_started":1}
I20260924 21:13:07.376802 17394 heartbeater.cc:347] Connected to a master server at 127.6.210.190:33035
I20260924 21:13:07.376932 17394 heartbeater.cc:464] Registering TS with master...
I20260924 21:13:07.377125 17394 heartbeater.cc:511] Master 127.6.210.190:33035 requested a full tablet report, sending...
I20260924 21:13:07.377717 17203 ts_manager.cc:194] Registered new tserver with Master: 9acde0d66caa449499a2d4f1debc14e8 (127.6.210.129:42681)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.130@KRBTEST.COM" created.
I20260924 21:13:07.378446 17203 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.129@KRBTEST.COM'} at 127.6.210.129:45985
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
I20260924 21:13:07.392844 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.130:0
--local_ip_for_outbound_sockets=127.6.210.130
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:33035
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:39951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:13:07.476451 17399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:13:07.476709 17399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:13:07.476752 17399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:13:07.477911 17399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:13:07.477982 17399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:13:07.478029 17399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.130
I20260924 21:13:07.479579 17399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:39951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:33035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.130
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:13:07.479859 17399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:13:07.480110 17399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:13:07.485260 17405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:13:07.485249 17407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:13:07.485249 17404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:13:07.485486 17399 server_base.cc:1034] running on GCE node
I20260924 21:13:07.485631 17399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:13:07.485827 17399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:13:07.486992 17399 hybrid_clock.cc:648] HybridClock initialized: now 1790284387486977 us; error 27 us; skew 500 ppm
Sep 24 21:13:07 dist-test-slave-q276 krb5kdc[17158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284387, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:13:07.489604 17399 init.cc:377] Logged in from keytab as kudu/127.6.210.130@KRBTEST.COM (short username kudu)
I20260924 21:13:07.490022 17399 webserver.cc:460] Webserver started at http://127.6.210.130:46127/ using document root <none> and password file <none>
I20260924 21:13:07.490244 17399 fs_manager.cc:359] Metadata directory not provided
I20260924 21:13:07.490399 17399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:13:07.490561 17399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:13:07.491710 17399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/data/instance:
uuid: "4ea77e12a26342ce8ff105752f071da6"
format_stamp: "Formatted at 2026-09-24 21:13:07 on dist-test-slave-q276"
server_key: "b70b34d46d5a86e5f14af9debb5e1025"
server_key_iv: "6c274c118866ae9aa2c00d67b6c5da75"
server_key_version: "encryptionkey@0"
I20260924 21:13:07.492071 17399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance:
uuid: "4ea77e12a26342ce8ff105752f071da6"
format_stamp: "Formatted at 2026-09-24 21:13:07 on dist-test-slave-q276"
server_key: "b70b34d46d5a86e5f14af9debb5e1025"
server_key_iv: "6c274c118866ae9aa2c00d67b6c5da75"
server_key_version: "encryptionkey@0"
I20260924 21:13:07.493508 17399 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:13:07.494321 17414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:07.494525 17399 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:13:07.494639 17399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/wal
uuid: "4ea77e12a26342ce8ff105752f071da6"
format_stamp: "Formatted at 2026-09-24 21:13:07 on dist-test-slave-q276"
server_key: "b70b34d46d5a86e5f14af9debb5e1025"
server_key_iv: "6c274c118866ae9aa2c00d67b6c5da75"
server_key_version: "encryptionkey@0"
I20260924 21:13:07.494743 17399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:13:07.523000 17399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:13:07.525292 17399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:13:07.525471 17399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:13:07.525732 17399 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:13:07.526075 17399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:13:07.526198 17399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:07.526258 17399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:13:07.526362 17399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:07.532899 17399 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.130:46193
I20260924 21:13:07.532941 17527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.130:46193 every 8 connection(s)
I20260924 21:13:07.533325 17399 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
I20260924 21:13:07.538061 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 17399
I20260924 21:13:07.538156 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance
I20260924 21:13:07.538419 6986 external_mini_cluster.cc:1428] Setting key 9d211efe4770accfdb60d3f491743a0f
Sep 24 21:13:07 dist-test-slave-q276 krb5kdc[17158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284387, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:13:07.544333 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:07.534099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:55109 (local address 127.6.210.190:33035)
0924 21:13:07.534262 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:07.534265 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:07.534522 (+ 257us) server_negotiation.cc:407] Connection header received
0924 21:13:07.535105 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:07.535106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:07.535119 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:07.535177 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:07.536228 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.536831 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.537969 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.538238 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.538363 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:07.541303 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:13:07.541304 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:13:07.541305 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:13:07.541312 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:13:07.542809 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:13:07.543302 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:13:07.543304 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:13:07.543304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:13:07.543368 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:13:07.543647 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:13:07.543648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:13:07.543649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:13:07.543768 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:13:07.543851 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0924 21:13:07.544144 (+ 293us) server_negotiation.cc:299] Negotiation successful
0924 21:13:07.544226 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260924 21:13:07.544564 17530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:07.534014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:33035 (local address 127.6.210.130:55109)
0924 21:13:07.534379 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:07.534418 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:07.534982 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:07.535264 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:07.535268 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:07.535891 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:13:07.536089 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:07.536094 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.536940 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.536941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:07.537871 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:07.537874 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.538377 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.538378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:07.538548 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:07.539213 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:13:07.539224 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:13:07.541177 (+ 1953us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:13:07.542918 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:13:07.542925 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:13:07.542927 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:13:07.543201 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:13:07.543458 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:13:07.543459 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:13:07.543459 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:13:07.543578 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:13:07.543841 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:13:07.543847 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:13:07.543993 (+ 146us) client_negotiation.cc:769] Sending connection context
0924 21:13:07.544104 (+ 111us) client_negotiation.cc:240] Negotiation successful
0924 21:13:07.544274 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":127,"threads_started":1}
I20260924 21:13:07.545058 17528 heartbeater.cc:347] Connected to a master server at 127.6.210.190:33035
I20260924 21:13:07.545179 17528 heartbeater.cc:464] Registering TS with master...
I20260924 21:13:07.545359 17528 heartbeater.cc:511] Master 127.6.210.190:33035 requested a full tablet report, sending...
I20260924 21:13:07.545746 17203 ts_manager.cc:194] Registered new tserver with Master: 4ea77e12a26342ce8ff105752f071da6 (127.6.210.130:46193)
I20260924 21:13:07.546131 17203 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.130@KRBTEST.COM'} at 127.6.210.130:55109
WARNING: no policy specified for kudu/127.6.210.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
I20260924 21:13:07.565433 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.131:0
--local_ip_for_outbound_sockets=127.6.210.131
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:33035
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:39951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:13:07.646090 17533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:13:07.646356 17533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:13:07.646400 17533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:13:07.647610 17533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:13:07.647682 17533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:13:07.647729 17533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.131
I20260924 21:13:07.649232 17533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:39951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:33035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.131
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:13:07.649500 17533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:13:07.649742 17533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:13:07.654858 17538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:13:07.654860 17541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:13:07.654846 17539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:13:07.655133 17533 server_base.cc:1034] running on GCE node
I20260924 21:13:07.655351 17533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:13:07.655586 17533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:13:07.656733 17533 hybrid_clock.cc:648] HybridClock initialized: now 1790284387656707 us; error 46 us; skew 500 ppm
Sep 24 21:13:07 dist-test-slave-q276 krb5kdc[17158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284387, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:13:07.659554 17533 init.cc:377] Logged in from keytab as kudu/127.6.210.131@KRBTEST.COM (short username kudu)
I20260924 21:13:07.660053 17533 webserver.cc:460] Webserver started at http://127.6.210.131:35939/ using document root <none> and password file <none>
I20260924 21:13:07.660343 17533 fs_manager.cc:359] Metadata directory not provided
I20260924 21:13:07.660420 17533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:13:07.660589 17533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:13:07.661733 17533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/data/instance:
uuid: "88fc9b29c80846489faebc70a861869b"
format_stamp: "Formatted at 2026-09-24 21:13:07 on dist-test-slave-q276"
server_key: "4ac327a16627d276cdd06ce4b3046957"
server_key_iv: "9c36bc6823c8d8bb8d5a165b1c321a43"
server_key_version: "encryptionkey@0"
I20260924 21:13:07.662144 17533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance:
uuid: "88fc9b29c80846489faebc70a861869b"
format_stamp: "Formatted at 2026-09-24 21:13:07 on dist-test-slave-q276"
server_key: "4ac327a16627d276cdd06ce4b3046957"
server_key_iv: "9c36bc6823c8d8bb8d5a165b1c321a43"
server_key_version: "encryptionkey@0"
I20260924 21:13:07.663777 17533 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:13:07.664750 17548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:07.664958 17533 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:13:07.665071 17533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/wal
uuid: "88fc9b29c80846489faebc70a861869b"
format_stamp: "Formatted at 2026-09-24 21:13:07 on dist-test-slave-q276"
server_key: "4ac327a16627d276cdd06ce4b3046957"
server_key_iv: "9c36bc6823c8d8bb8d5a165b1c321a43"
server_key_version: "encryptionkey@0"
I20260924 21:13:07.665153 17533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:13:07.683068 17533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:13:07.685355 17533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:13:07.685515 17533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:13:07.685778 17533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:13:07.686127 17533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:13:07.686164 17533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:07.686189 17533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:13:07.686240 17533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:07.692896 17533 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.131:45441
I20260924 21:13:07.692941 17661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.131:45441 every 8 connection(s)
I20260924 21:13:07.693282 17533 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
Sep 24 21:13:07 dist-test-slave-q276 krb5kdc[17158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284387, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:13:07.700946 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 17533
I20260924 21:13:07.701047 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance
I20260924 21:13:07.701256 6986 external_mini_cluster.cc:1428] Setting key 60e90d8b4c0df85ce7fa46ce992e437d
I20260924 21:13:07.703462 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:07.694035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:57331 (local address 127.6.210.190:33035)
0924 21:13:07.694085 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:07.694091 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:07.694337 (+ 246us) server_negotiation.cc:407] Connection header received
0924 21:13:07.694916 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:07.694917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:07.694928 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:07.694988 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:07.695977 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.696542 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.697633 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.697907 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.697968 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:07.700274 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:13:07.700276 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:13:07.700277 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:13:07.700288 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:13:07.702225 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:13:07.702623 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:13:07.702623 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:13:07.702624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:13:07.702680 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:13:07.702916 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:13:07.702917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:13:07.702917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:13:07.703027 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:13:07.703081 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 21:13:07.703329 (+ 248us) server_negotiation.cc:299] Negotiation successful
0924 21:13:07.703395 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 21:13:07.703627 17664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:07.694003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:33035 (local address 127.6.210.131:57331)
0924 21:13:07.694168 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:07.694213 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:07.694812 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:07.695048 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:07.695052 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:07.695656 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:13:07.695872 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:07.695877 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.696635 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.696636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:07.697543 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:07.697545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.697973 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.697973 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:07.698081 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:07.698597 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:13:07.698605 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:13:07.700145 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:13:07.702323 (+ 2178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:13:07.702329 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:13:07.702331 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:13:07.702554 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:13:07.702762 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:13:07.702763 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:13:07.702763 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:13:07.702857 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:13:07.703078 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:13:07.703082 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:13:07.703192 (+ 110us) client_negotiation.cc:769] Sending connection context
0924 21:13:07.703295 (+ 103us) client_negotiation.cc:240] Negotiation successful
0924 21:13:07.703411 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":97,"threads_started":1}
I20260924 21:13:07.704077 17662 heartbeater.cc:347] Connected to a master server at 127.6.210.190:33035
I20260924 21:13:07.704180 17662 heartbeater.cc:464] Registering TS with master...
I20260924 21:13:07.704331 17662 heartbeater.cc:511] Master 127.6.210.190:33035 requested a full tablet report, sending...
I20260924 21:13:07.704725 17203 ts_manager.cc:194] Registered new tserver with Master: 88fc9b29c80846489faebc70a861869b (127.6.210.131:45441)
I20260924 21:13:07.705096 17203 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.131@KRBTEST.COM'} at 127.6.210.131:57331
I20260924 21:13:07.713135 6986 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:13:07.721441 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:07.714600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43548 (local address 127.6.210.190:33035)
0924 21:13:07.714671 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:07.714674 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:07.714785 (+ 111us) server_negotiation.cc:407] Connection header received
0924 21:13:07.714884 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:07.714885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:07.714899 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:07.714974 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:07.715584 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.716083 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.716941 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.717138 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.717235 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:07.718641 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:13:07.718643 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:13:07.718643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:13:07.718650 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:13:07.720164 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:13:07.720555 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:13:07.720556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:13:07.720557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:13:07.720623 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:13:07.720841 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:13:07.720842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:13:07.720842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:13:07.721003 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:13:07.721116 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0924 21:13:07.721246 (+ 130us) server_negotiation.cc:299] Negotiation successful
0924 21:13:07.721331 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260924 21:13:07.723245 17203 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43548:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:13:07.724757 17203 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:13:07.731470 17673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:07.727070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:42681 (local address 127.0.0.1:43260)
0924 21:13:07.727170 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:07.727212 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:07.727273 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:07.727868 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:07.727871 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:07.727874 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:13:07.727956 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:07.727960 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.729324 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.729325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:07.730381 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:07.730383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.731109 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.731110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:07.731202 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:07.731217 (+ 15us) client_negotiation.cc:769] Sending connection context
0924 21:13:07.731304 (+ 87us) client_negotiation.cc:240] Negotiation successful
0924 21:13:07.731371 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76,"mutex_wait_us":1,"thread_start_us":65,"threads_started":1}
I20260924 21:13:07.731477 17676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:07.727277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:45441 (local address 127.0.0.1:41852)
0924 21:13:07.727457 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:07.727470 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:07.727514 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:07.727966 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:07.727967 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:07.727970 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:13:07.728020 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:07.728022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.728987 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.728989 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:07.730358 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:07.730361 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.731247 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.731248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:07.731307 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:07.731311 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:13:07.731375 (+ 64us) client_negotiation.cc:240] Negotiation successful
0924 21:13:07.731409 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":1,"thread_start_us":127,"threads_started":1}
I20260924 21:13:07.731527 17678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:07.727532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41852 (local address 127.6.210.131:45441)
0924 21:13:07.727690 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:07.727694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:07.727703 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:13:07.727739 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:07.727740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:07.727753 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:07.727876 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:13:07.728108 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.728861 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.730463 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.731173 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.731248 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:07.731307 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 21:13:07.731388 (+ 81us) server_negotiation.cc:299] Negotiation successful
0924 21:13:07.731426 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":76,"threads_started":1}
I20260924 21:13:07.731920 17675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:07.727287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43260 (local address 127.6.210.129:42681)
0924 21:13:07.727651 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:07.727657 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:07.727669 (+ 12us) server_negotiation.cc:407] Connection header received
0924 21:13:07.727712 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:07.727715 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:07.727735 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:07.728033 (+ 298us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:13:07.728104 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.729213 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.730500 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.730999 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.731106 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:07.731172 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 21:13:07.731753 (+ 581us) server_negotiation.cc:299] Negotiation successful
0924 21:13:07.731825 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":83,"threads_started":1}
I20260924 21:13:07.732388 17596 tablet_service.cc:1467] Processing CreateTablet for tablet 474f67e5283e4e00805143eafadc198e (DEFAULT_TABLE table=test-table [id=b78ae306d1f944c4b839885a92092751]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:13:07.732725 17596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 474f67e5283e4e00805143eafadc198e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:13:07.732645 17328 tablet_service.cc:1467] Processing CreateTablet for tablet 474f67e5283e4e00805143eafadc198e (DEFAULT_TABLE table=test-table [id=b78ae306d1f944c4b839885a92092751]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:13:07.732947 17674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:07.727178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:46193 (local address 127.0.0.1:49448)
0924 21:13:07.727437 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:07.727449 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:07.727508 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:07.727894 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:07.727894 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:07.727896 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:13:07.727956 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:07.727958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.730310 (+ 2352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.730311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:07.731455 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:07.731458 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.732677 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.732679 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:07.732743 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:07.732748 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 21:13:07.732833 (+ 85us) client_negotiation.cc:240] Negotiation successful
0924 21:13:07.732876 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":6,"thread_start_us":100,"threads_started":1}
I20260924 21:13:07.732975 17328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 474f67e5283e4e00805143eafadc198e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:13:07.732988 17677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:07.727439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49448 (local address 127.6.210.130:46193)
0924 21:13:07.727712 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:07.727716 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:07.727725 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:13:07.727783 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:07.727785 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:07.727799 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:07.728033 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:13:07.728102 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.729213 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.731551 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.732578 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.732657 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:07.732711 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 21:13:07.732856 (+ 145us) server_negotiation.cc:299] Negotiation successful
0924 21:13:07.732903 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":82,"threads_started":1}
I20260924 21:13:07.735438 17462 tablet_service.cc:1467] Processing CreateTablet for tablet 474f67e5283e4e00805143eafadc198e (DEFAULT_TABLE table=test-table [id=b78ae306d1f944c4b839885a92092751]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:13:07.735764 17679 tablet_bootstrap.cc:492] T 474f67e5283e4e00805143eafadc198e P 9acde0d66caa449499a2d4f1debc14e8: Bootstrap starting.
I20260924 21:13:07.735790 17462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 474f67e5283e4e00805143eafadc198e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:13:07.736397 17679 tablet_bootstrap.cc:654] T 474f67e5283e4e00805143eafadc198e P 9acde0d66caa449499a2d4f1debc14e8: Neither blocks nor log segments found. Creating new log.
I20260924 21:13:07.736866 17679 log.cc:824] T 474f67e5283e4e00805143eafadc198e P 9acde0d66caa449499a2d4f1debc14e8: Log is configured to *not* fsync() on all Append() calls
I20260924 21:13:07.737232 17680 tablet_bootstrap.cc:492] T 474f67e5283e4e00805143eafadc198e P 88fc9b29c80846489faebc70a861869b: Bootstrap starting.
I20260924 21:13:07.737656 17679 tablet_bootstrap.cc:492] T 474f67e5283e4e00805143eafadc198e P 9acde0d66caa449499a2d4f1debc14e8: No bootstrap required, opened a new log
I20260924 21:13:07.737720 17679 ts_tablet_manager.cc:1397] T 474f67e5283e4e00805143eafadc198e P 9acde0d66caa449499a2d4f1debc14e8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:13:07.737906 17680 tablet_bootstrap.cc:654] T 474f67e5283e4e00805143eafadc198e P 88fc9b29c80846489faebc70a861869b: Neither blocks nor log segments found. Creating new log.
I20260924 21:13:07.738436 17680 log.cc:824] T 474f67e5283e4e00805143eafadc198e P 88fc9b29c80846489faebc70a861869b: Log is configured to *not* fsync() on all Append() calls
I20260924 21:13:07.739259 17680 tablet_bootstrap.cc:492] T 474f67e5283e4e00805143eafadc198e P 88fc9b29c80846489faebc70a861869b: No bootstrap required, opened a new log
I20260924 21:13:07.739331 17680 ts_tablet_manager.cc:1397] T 474f67e5283e4e00805143eafadc198e P 88fc9b29c80846489faebc70a861869b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:13:07.739861 17679 raft_consensus.cc:348] T 474f67e5283e4e00805143eafadc198e P 9acde0d66caa449499a2d4f1debc14e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9acde0d66caa449499a2d4f1debc14e8" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42681 } } peers { permanent_uuid: "88fc9b29c80846489faebc70a861869b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 45441 } } peers { permanent_uuid: "4ea77e12a26342ce8ff105752f071da6" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 46193 } }
I20260924 21:13:07.740041 17679 raft_consensus.cc:374] T 474f67e5283e4e00805143eafadc198e P 9acde0d66caa449499a2d4f1debc14e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:13:07.740077 17679 raft_consensus.cc:729] T 474f67e5283e4e00805143eafadc198e P 9acde0d66caa449499a2d4f1debc14e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9acde0d66caa449499a2d4f1debc14e8, State: Initialized, Role: FOLLOWER
I20260924 21:13:07.740190 17679 consensus_queue.cc:260] T 474f67e5283e4e00805143eafadc198e P 9acde0d66caa449499a2d4f1debc14e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9acde0d66caa449499a2d4f1debc14e8" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42681 } } peers { permanent_uuid: "88fc9b29c80846489faebc70a861869b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 45441 } } peers { permanent_uuid: "4ea77e12a26342ce8ff105752f071da6" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 46193 } }
I20260924 21:13:07.740459 17679 ts_tablet_manager.cc:1428] T 474f67e5283e4e00805143eafadc198e P 9acde0d66caa449499a2d4f1debc14e8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:13:07.740504 17394 heartbeater.cc:503] Master 127.6.210.190:33035 was elected leader, sending a full tablet report...
I20260924 21:13:07.741174 17680 raft_consensus.cc:348] T 474f67e5283e4e00805143eafadc198e P 88fc9b29c80846489faebc70a861869b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9acde0d66caa449499a2d4f1debc14e8" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42681 } } peers { permanent_uuid: "88fc9b29c80846489faebc70a861869b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 45441 } } peers { permanent_uuid: "4ea77e12a26342ce8ff105752f071da6" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 46193 } }
I20260924 21:13:07.741344 17680 raft_consensus.cc:374] T 474f67e5283e4e00805143eafadc198e P 88fc9b29c80846489faebc70a861869b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:13:07.741389 17680 raft_consensus.cc:729] T 474f67e5283e4e00805143eafadc198e P 88fc9b29c80846489faebc70a861869b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88fc9b29c80846489faebc70a861869b, State: Initialized, Role: FOLLOWER
I20260924 21:13:07.741503 17680 consensus_queue.cc:260] T 474f67e5283e4e00805143eafadc198e P 88fc9b29c80846489faebc70a861869b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9acde0d66caa449499a2d4f1debc14e8" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42681 } } peers { permanent_uuid: "88fc9b29c80846489faebc70a861869b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 45441 } } peers { permanent_uuid: "4ea77e12a26342ce8ff105752f071da6" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 46193 } }
I20260924 21:13:07.741797 17680 ts_tablet_manager.cc:1428] T 474f67e5283e4e00805143eafadc198e P 88fc9b29c80846489faebc70a861869b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 21:13:07.741837 17662 heartbeater.cc:503] Master 127.6.210.190:33035 was elected leader, sending a full tablet report...
I20260924 21:13:07.741902 17683 tablet_bootstrap.cc:492] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6: Bootstrap starting.
I20260924 21:13:07.742508 17683 tablet_bootstrap.cc:654] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6: Neither blocks nor log segments found. Creating new log.
I20260924 21:13:07.743079 17683 log.cc:824] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6: Log is configured to *not* fsync() on all Append() calls
I20260924 21:13:07.743695 17683 tablet_bootstrap.cc:492] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6: No bootstrap required, opened a new log
I20260924 21:13:07.743750 17683 ts_tablet_manager.cc:1397] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260924 21:13:07.744992 17683 raft_consensus.cc:348] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9acde0d66caa449499a2d4f1debc14e8" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42681 } } peers { permanent_uuid: "88fc9b29c80846489faebc70a861869b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 45441 } } peers { permanent_uuid: "4ea77e12a26342ce8ff105752f071da6" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 46193 } }
I20260924 21:13:07.745103 17683 raft_consensus.cc:374] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:13:07.745124 17683 raft_consensus.cc:729] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ea77e12a26342ce8ff105752f071da6, State: Initialized, Role: FOLLOWER
I20260924 21:13:07.745226 17683 consensus_queue.cc:260] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9acde0d66caa449499a2d4f1debc14e8" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42681 } } peers { permanent_uuid: "88fc9b29c80846489faebc70a861869b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 45441 } } peers { permanent_uuid: "4ea77e12a26342ce8ff105752f071da6" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 46193 } }
I20260924 21:13:07.745477 17683 ts_tablet_manager.cc:1428] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:13:07.745574 17528 heartbeater.cc:503] Master 127.6.210.190:33035 was elected leader, sending a full tablet report...
W20260924 21:13:07.783916 17529 tablet.cc:2369] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:13:07.809813 17687 raft_consensus.cc:482] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:13:07.809940 17687 raft_consensus.cc:504] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9acde0d66caa449499a2d4f1debc14e8" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42681 } } peers { permanent_uuid: "88fc9b29c80846489faebc70a861869b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 45441 } } peers { permanent_uuid: "4ea77e12a26342ce8ff105752f071da6" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 46193 } }
I20260924 21:13:07.810417 17687 leader_election.cc:288] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9acde0d66caa449499a2d4f1debc14e8 (127.6.210.129:42681), 88fc9b29c80846489faebc70a861869b (127.6.210.131:45441)
I20260924 21:13:07.813493 17675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:07.810858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:49215 (local address 127.6.210.129:42681)
0924 21:13:07.810950 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:07.810952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:07.810963 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:13:07.810994 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:07.810995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:07.811004 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:07.811073 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:13:07.811237 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.811730 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.812699 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.813083 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.813171 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:07.813180 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 21:13:07.813379 (+ 199us) server_negotiation.cc:299] Negotiation successful
0924 21:13:07.813417 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260924 21:13:07.813527 17530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:07.810622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:42681 (local address 127.6.210.130:49215)
0924 21:13:07.810684 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:07.810697 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:07.810743 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:07.811080 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:07.811081 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:07.811083 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:13:07.811159 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:07.811160 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.811824 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.811826 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:07.812596 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:07.812598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.813167 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.813168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:07.813315 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:07.813319 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:13:07.813373 (+ 54us) client_negotiation.cc:240] Negotiation successful
0924 21:13:07.813414 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260924 21:13:07.813537 17678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:07.810960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:40349 (local address 127.6.210.131:45441)
0924 21:13:07.811015 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:07.811017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:07.811028 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:13:07.811058 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:07.811059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:07.811070 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:07.811140 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:13:07.811355 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.811822 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.812725 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.813165 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.813240 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:07.813248 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:13:07.813434 (+ 186us) server_negotiation.cc:299] Negotiation successful
0924 21:13:07.813469 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 21:13:07.813643 17688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:07.810785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:45441 (local address 127.6.210.130:40349)
0924 21:13:07.810954 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:07.810963 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:07.811009 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:07.811217 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:07.811218 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:07.811220 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:13:07.811285 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:07.811286 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.811905 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.811906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:07.812637 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:07.812639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.813252 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.813253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:07.813353 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:07.813356 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:13:07.813418 (+ 62us) client_negotiation.cc:240] Negotiation successful
0924 21:13:07.813458 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":125,"threads_started":1}
I20260924 21:13:07.813913 17348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "474f67e5283e4e00805143eafadc198e" candidate_uuid: "4ea77e12a26342ce8ff105752f071da6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9acde0d66caa449499a2d4f1debc14e8" is_pre_election: true
I20260924 21:13:07.813922 17616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "474f67e5283e4e00805143eafadc198e" candidate_uuid: "4ea77e12a26342ce8ff105752f071da6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88fc9b29c80846489faebc70a861869b" is_pre_election: true
I20260924 21:13:07.814092 17616 raft_consensus.cc:2454] T 474f67e5283e4e00805143eafadc198e P 88fc9b29c80846489faebc70a861869b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ea77e12a26342ce8ff105752f071da6 in term 0.
I20260924 21:13:07.814095 17348 raft_consensus.cc:2454] T 474f67e5283e4e00805143eafadc198e P 9acde0d66caa449499a2d4f1debc14e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ea77e12a26342ce8ff105752f071da6 in term 0.
I20260924 21:13:07.814334 17416 leader_election.cc:302] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ea77e12a26342ce8ff105752f071da6, 88fc9b29c80846489faebc70a861869b; no voters:
I20260924 21:13:07.814456 17687 raft_consensus.cc:2781] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:13:07.814510 17687 raft_consensus.cc:482] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:13:07.814538 17687 raft_consensus.cc:3037] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:13:07.815227 17687 raft_consensus.cc:504] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9acde0d66caa449499a2d4f1debc14e8" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42681 } } peers { permanent_uuid: "88fc9b29c80846489faebc70a861869b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 45441 } } peers { permanent_uuid: "4ea77e12a26342ce8ff105752f071da6" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 46193 } }
I20260924 21:13:07.815392 17687 leader_election.cc:288] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6 [CANDIDATE]: Term 1 election: Requested vote from peers 9acde0d66caa449499a2d4f1debc14e8 (127.6.210.129:42681), 88fc9b29c80846489faebc70a861869b (127.6.210.131:45441)
I20260924 21:13:07.815577 17616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "474f67e5283e4e00805143eafadc198e" candidate_uuid: "4ea77e12a26342ce8ff105752f071da6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88fc9b29c80846489faebc70a861869b"
I20260924 21:13:07.815610 17348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "474f67e5283e4e00805143eafadc198e" candidate_uuid: "4ea77e12a26342ce8ff105752f071da6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9acde0d66caa449499a2d4f1debc14e8"
I20260924 21:13:07.815677 17616 raft_consensus.cc:3037] T 474f67e5283e4e00805143eafadc198e P 88fc9b29c80846489faebc70a861869b [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:13:07.815677 17348 raft_consensus.cc:3037] T 474f67e5283e4e00805143eafadc198e P 9acde0d66caa449499a2d4f1debc14e8 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:13:07.816555 17616 raft_consensus.cc:2454] T 474f67e5283e4e00805143eafadc198e P 88fc9b29c80846489faebc70a861869b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ea77e12a26342ce8ff105752f071da6 in term 1.
I20260924 21:13:07.816555 17348 raft_consensus.cc:2454] T 474f67e5283e4e00805143eafadc198e P 9acde0d66caa449499a2d4f1debc14e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ea77e12a26342ce8ff105752f071da6 in term 1.
I20260924 21:13:07.816749 17416 leader_election.cc:302] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ea77e12a26342ce8ff105752f071da6, 9acde0d66caa449499a2d4f1debc14e8; no voters:
I20260924 21:13:07.816846 17687 raft_consensus.cc:2781] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:13:07.817016 17687 raft_consensus.cc:686] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6 [term 1 LEADER]: Becoming Leader. State: Replica: 4ea77e12a26342ce8ff105752f071da6, State: Running, Role: LEADER
I20260924 21:13:07.817112 17687 consensus_queue.cc:237] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9acde0d66caa449499a2d4f1debc14e8" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42681 } } peers { permanent_uuid: "88fc9b29c80846489faebc70a861869b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 45441 } } peers { permanent_uuid: "4ea77e12a26342ce8ff105752f071da6" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 46193 } }
I20260924 21:13:07.817893 17203 catalog_manager.cc:5495] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ea77e12a26342ce8ff105752f071da6 (127.6.210.130). New cstate: current_term: 1 leader_uuid: "4ea77e12a26342ce8ff105752f071da6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9acde0d66caa449499a2d4f1debc14e8" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88fc9b29c80846489faebc70a861869b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 45441 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ea77e12a26342ce8ff105752f071da6" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 46193 } health_report { overall_health: HEALTHY } } }
I20260924 21:13:07.825914 17677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:07.823876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49454 (local address 127.6.210.130:46193)
0924 21:13:07.823933 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:07.823934 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:07.823941 (+ 7us) server_negotiation.cc:407] Connection header received
0924 21:13:07.823968 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:07.823968 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:07.823983 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:07.824049 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:13:07.824182 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.824614 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.825300 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:07.825472 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:07.825522 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:07.825639 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:13:07.825729 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:13:07.825787 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 21:13:07.825832 (+ 45us) server_negotiation.cc:299] Negotiation successful
0924 21:13:07.825848 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 21:13:07.827642 17348 raft_consensus.cc:1261] T 474f67e5283e4e00805143eafadc198e P 9acde0d66caa449499a2d4f1debc14e8 [term 1 FOLLOWER]: Refusing update from remote peer 4ea77e12a26342ce8ff105752f071da6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:13:07.827666 17616 raft_consensus.cc:1261] T 474f67e5283e4e00805143eafadc198e P 88fc9b29c80846489faebc70a861869b [term 1 FOLLOWER]: Refusing update from remote peer 4ea77e12a26342ce8ff105752f071da6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:13:07.828023 17687 consensus_queue.cc:1035] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88fc9b29c80846489faebc70a861869b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 45441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:13:07.828156 17689 consensus_queue.cc:1035] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9acde0d66caa449499a2d4f1debc14e8" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 42681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:13:07.830518 17692 mvcc.cc:205] Tried to move back new op lower bound from 7333004852539404288 to 7333004852499415040. Current Snapshot: MvccSnapshot[applied={T|T < 7333004852539404288}]
I20260924 21:13:07.830751 17693 mvcc.cc:205] Tried to move back new op lower bound from 7333004852539404288 to 7333004852499415040. Current Snapshot: MvccSnapshot[applied={T|T < 7333004852539404288}]
I20260924 21:13:07.831489 17694 mvcc.cc:205] Tried to move back new op lower bound from 7333004852539404288 to 7333004852499415040. Current Snapshot: MvccSnapshot[applied={T|T < 7333004852539404288}]
I20260924 21:13:07.832828 17202 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:13:07.832938 17202 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:13:07.833683 17202 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 550f4b53a02248ae98aeec768aafa75b: Sending DeleteTablet for 3 replicas of tablet 474f67e5283e4e00805143eafadc198e
I20260924 21:13:07.834028 17596 tablet_service.cc:1514] Processing DeleteTablet for tablet 474f67e5283e4e00805143eafadc198e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:13:07 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:41852
I20260924 21:13:07.834049 17328 tablet_service.cc:1514] Processing DeleteTablet for tablet 474f67e5283e4e00805143eafadc198e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:13:07 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:43260
I20260924 21:13:07.834285 17703 tablet_replica.cc:330] T 474f67e5283e4e00805143eafadc198e P 9acde0d66caa449499a2d4f1debc14e8: stopping tablet replica
I20260924 21:13:07.834244 17462 tablet_service.cc:1514] Processing DeleteTablet for tablet 474f67e5283e4e00805143eafadc198e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:13:07 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:49448
I20260924 21:13:07.834384 17703 raft_consensus.cc:2229] T 474f67e5283e4e00805143eafadc198e P 9acde0d66caa449499a2d4f1debc14e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:13:07.834448 17704 tablet_replica.cc:330] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6: stopping tablet replica
I20260924 21:13:07.834498 17703 raft_consensus.cc:2258] T 474f67e5283e4e00805143eafadc198e P 9acde0d66caa449499a2d4f1debc14e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:13:07.834525 17704 raft_consensus.cc:2229] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6 [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:13:07.834576 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 17265
I20260924 21:13:07.834650 17702 tablet_replica.cc:330] T 474f67e5283e4e00805143eafadc198e P 88fc9b29c80846489faebc70a861869b: stopping tablet replica
I20260924 21:13:07.834743 17702 raft_consensus.cc:2229] T 474f67e5283e4e00805143eafadc198e P 88fc9b29c80846489faebc70a861869b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:13:07.834833 17702 raft_consensus.cc:2258] T 474f67e5283e4e00805143eafadc198e P 88fc9b29c80846489faebc70a861869b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:13:07.834645 17704 raft_consensus.cc:2258] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:13:07.835577 17702 ts_tablet_manager.cc:1905] T 474f67e5283e4e00805143eafadc198e P 88fc9b29c80846489faebc70a861869b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:13:07.835711 17704 ts_tablet_manager.cc:1905] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:13:07.837236 17704 ts_tablet_manager.cc:1918] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:13:07.837339 17704 log.cc:1196] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-1/wal/wals/474f67e5283e4e00805143eafadc198e
I20260924 21:13:07.837538 17702 ts_tablet_manager.cc:1918] T 474f67e5283e4e00805143eafadc198e P 88fc9b29c80846489faebc70a861869b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:13:07.837613 17702 log.cc:1196] T 474f67e5283e4e00805143eafadc198e P 88fc9b29c80846489faebc70a861869b: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284333163253-6986-0/minicluster-data/ts-2/wal/wals/474f67e5283e4e00805143eafadc198e
I20260924 21:13:07.837628 17704 ts_tablet_manager.cc:1939] T 474f67e5283e4e00805143eafadc198e P 4ea77e12a26342ce8ff105752f071da6: Deleting consensus metadata
I20260924 21:13:07.837886 17702 ts_tablet_manager.cc:1939] T 474f67e5283e4e00805143eafadc198e P 88fc9b29c80846489faebc70a861869b: Deleting consensus metadata
I20260924 21:13:07.838034 17189 catalog_manager.cc:4841] TS 4ea77e12a26342ce8ff105752f071da6 (127.6.210.130:46193): tablet 474f67e5283e4e00805143eafadc198e (table test-table [id=b78ae306d1f944c4b839885a92092751]) successfully deleted
I20260924 21:13:07.838254 17188 catalog_manager.cc:4841] TS 88fc9b29c80846489faebc70a861869b (127.6.210.131:45441): tablet 474f67e5283e4e00805143eafadc198e (table test-table [id=b78ae306d1f944c4b839885a92092751]) successfully deleted
W20260924 21:13:07.841322 17188 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.210.129:42681 (error 108)
I20260924 21:13:07.841639 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 17399
I20260924 21:13:07.841861 17674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:07.841731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:42681 (local address 127.0.0.1:43276)
0924 21:13:07.841789 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:07.841828 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.129:42681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260924 21:13:07.841964 17188 catalog_manager.cc:4584] TS 9acde0d66caa449499a2d4f1debc14e8 (127.6.210.129:42681): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 474f67e5283e4e00805143eafadc198e: Network error: Client connection negotiation failed: client connection to 127.6.210.129:42681: connect: Connection refused (error 111)
I20260924 21:13:07.846670 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 17533
I20260924 21:13:07.851258 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 17171
2026-09-24T21:13:07Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1948 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:13:07 dist-test-slave-q276 krb5kdc[17708](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:13:07 dist-test-slave-q276 krb5kdc[17708](info): set up 2 sockets
Sep 24 21:13:07 dist-test-slave-q276 krb5kdc[17708](info): commencing operation
krb5kdc: starting...
W20260924 21:13:08.963248 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:13:09 dist-test-slave-q276 krb5kdc[17708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:13:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:13:09Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.6.210.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.6.210.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.6.210.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/rangeradmin_127.6.210.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.6.210.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/rangeradmin_127.6.210.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.6.210.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.6.210.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.6.210.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/rangerlookup_127.6.210.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.6.210.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/rangerlookup_127.6.210.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.210.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.6.210.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.210.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/HTTP_127.6.210.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.210.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/HTTP_127.6.210.148@KRBTEST.COM.keytab.
I20260924 21:13:09.129426 6986 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task5s070Q/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/postgres -l logfile start
2026-09-24 21:13:11.741 UTC [17738] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-24 21:13:11.741 UTC [17738] LOG: listening on IPv4 address "127.6.210.148", port 37151
2026-09-24 21:13:11.744 UTC [17738] LOG: listening on Unix socket "/tmp/.s.PGSQL.37151"
2026-09-24 21:13:11.779 UTC [17741] LOG: database system was shut down at 2026-09-24 21:13:10 UTC
2026-09-24 21:13:11.783 UTC [17738] LOG: database system is ready to accept connections
I20260924 21:13:12.779521 6986 mini_postgres.cc:96] Postgres bound to 37151
2026-09-24 21:13:12.785 UTC [17749] FATAL: database "slave" does not exist
127.6.210.148:37151 - accepting connections
I20260924 21:13:12.786090 6986 mini_ranger.cc:161] Starting Ranger...
I20260924 21:13:12.800969 6986 mini_ranger.cc:84] Created miniranger Postgres user
I20260924 21:13:13.293063 6986 mini_ranger.cc:87] Created ranger Postgres database
I20260924 21:13:13.293150 6986 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ranger-admin
2026-09-24 21:13:13,554 [I] DB FLAVOR :POSTGRES
2026-09-24 21:13:13,555 [I] --------- Verifying Ranger DB connection ---------
2026-09-24 21:13:13,555 [I] Checking connection..
2026-09-24 21:13:13,555 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:37151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-24 21:13:13,834 [I] Checking connection passed.
2026-09-24 21:13:13,834 [I] --------- Verifying version history table ---------
2026-09-24 21:13:13,834 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:37151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 21:13:14,114 [I] Table x_db_version_h does not exist in database ranger
2026-09-24 21:13:14,114 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:37151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 21:13:14,360 [I] Table x_db_version_h does not exist in database ranger
2026-09-24 21:13:14,361 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-24 21:13:14,361 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:37151/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-24 21:13:14.616 UTC [17813] WARNING: there is no transaction in progress
2026-09-24 21:13:14,622 [I] create_dbversion_catalog.sql file imported successfully
2026-09-24 21:13:14,623 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:37151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 21:13:14,867 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-24 21:13:14,867 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-24 21:13:14,867 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:37151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-24 21:13:15,134 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:37151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-24 21:13:15,392 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:37151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-q276.c.gcp-upstream.internal','N') ;"
2026-09-24 21:13:15,638 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:37151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-24 21:13:15,882 [I] Table x_portal_user does not exist in database ranger
2026-09-24 21:13:15,883 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:37151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-24 21:13:16,122 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-24 21:13:16,123 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:37151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-24 21:13:16,355 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:37151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-24 21:13:16,616 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-24 21:13:16,616 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:37151/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-24 21:13:17.595 UTC [17921] WARNING: there is no transaction in progress
2026-09-24 21:13:17.612 UTC [17921] WARNING: there is no transaction in progress
2026-09-24 21:13:17.628 UTC [17921] WARNING: there is no transaction in progress
2026-09-24 21:13:17.645 UTC [17921] WARNING: there is no transaction in progress
2026-09-24 21:13:17.665 UTC [17921] WARNING: there is no transaction in progress
2026-09-24 21:13:17.685 UTC [17921] WARNING: there is no transaction in progress
2026-09-24 21:13:17.746 UTC [17921] WARNING: there is no transaction in progress
2026-09-24 21:13:17.753 UTC [17921] WARNING: there is no transaction in progress
2026-09-24 21:13:17.768 UTC [17921] WARNING: there is no transaction in progress
2026-09-24 21:13:17.778 UTC [17921] WARNING: there is no transaction in progress
2026-09-24 21:13:17.788 UTC [17921] WARNING: there is no transaction in progress
2026-09-24 21:13:17.802 UTC [17921] WARNING: there is no transaction in progress
2026-09-24 21:13:17.813 UTC [17921] WARNING: there is no transaction in progress
2026-09-24 21:13:17.820 UTC [17921] WARNING: there is no transaction in progress
2026-09-24 21:13:17.827 UTC [17921] WARNING: there is no transaction in progress
2026-09-24 21:13:17.833 UTC [17921] WARNING: there is no transaction in progress
2026-09-24 21:13:18,474 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-24 21:13:18,475 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:37151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-24 21:13:18,708 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-24 21:13:18,708 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:37151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-24 21:13:18,994 [I] Table x_portal_user already exists in database 'ranger'
2026-09-24 21:13:18,994 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:37151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-24 21:13:19,262 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-24 21:13:19,262 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:37151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-24 21:13:19,501 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:37151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-24 21:13:19,728 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:37151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-q276.c.gcp-upstream.internal';"
2026-09-24 21:13:19,975 [I] CORE_DB_SCHEMA import status has been updated
2026-09-24 21:13:19,976 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:37151/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-24 21:13:20,241 [I] DB_PATCHES have already been applied
I20260924 21:13:20.247027 6986 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ranger-admin:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task5s070Q/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260924 21:13:20.247108 6986 mini_ranger.cc:193] Using host: 127.6.210.148
I20260924 21:13:20.249941 6986 mini_ranger.cc:227] Ranger admin URL: http://127.6.210.148:50957
Sep 24, 2026 9:13:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 24, 2026 9:13:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 24, 2026 9:13:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ranger-admin/ews/webapp
Sep 24, 2026 9:13:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 24, 2026 9:13:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 24, 2026 9:13:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ranger-admin/ews/webapp].
Sep 24, 2026 9:13:21 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260924 21:13:21.359145 6986 mini_ranger.cc:160] Time spent starting Ranger: real 8.573s user 0.000s sys 0.009s
Sep 24, 2026 9:13:40 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 24, 2026 9:13:40 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 24, 2026 9:13:40 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 24, 2026 9:13:40 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 24, 2026 9:13:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 24, 2026 9:13:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 24, 2026 9:13:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 24, 2026 9:13:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 24, 2026 9:13:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 24, 2026 9:13:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 24, 2026 9:13:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 24, 2026 9:13:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 24, 2026 9:13:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 24, 2026 9:13:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 24, 2026 9:13:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 24, 2026 9:13:40 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 24, 2026 9:13:40 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 24, 2026 9:13:41 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260924 21:13:42.454341 6986 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.6.210.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.6.210.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.6.210.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/rangerkms_127.6.210.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.6.210.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/rangerkms_127.6.210.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.210.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.6.210.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.210.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/HTTP_127.6.210.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.210.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/HTTP_127.6.210.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 24 21:13:42 dist-test-slave-q276 krb5kdc[17708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284422, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260924 21:13:42.540558 6986 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260924 21:13:42.550918 6986 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260924 21:13:43.446846 6986 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260924 21:13:43.446929 6986 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ranger-kms
I20260924 21:13:43.446952 6986 mini_ranger_kms.cc:227] Using postgres at 127.6.210.148:37151
2026-09-24 21:13:43,619 [I] DB FLAVOR :POSTGRES
2026-09-24 21:13:43,619 [I] --------- Verifying Ranger DB connection ---------
2026-09-24 21:13:43,619 [I] Checking connection
2026-09-24 21:13:43,619 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:37151/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-24 21:13:43,858 [I] connection success
2026-09-24 21:13:43,859 [I] --------- Verifying Ranger DB tables ---------
2026-09-24 21:13:43,859 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-24 21:13:43,859 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:37151/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-24 21:13:44,110 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-24 21:13:44,111 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-24 21:13:44,111 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-24 21:13:44,111 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:37151/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-24 21:13:44,385 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260924 21:13:44.830415 6986 mini_ranger_kms.cc:310] Created kms service
I20260924 21:13:45.012122 6986 mini_ranger_kms.cc:326] Created kudu user
I20260924 21:13:45.053746 6986 mini_ranger_kms.cc:343] Created rangerkms user
Sep 24 21:13:45 dist-test-slave-q276 krb5kdc[17708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284422, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.6.210.148@KRBTEST.COM
I20260924 21:13:45.414903 6986 mini_ranger_kms.cc:384] Added ranger policy
I20260924 21:13:45.415052 6986 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ranger-kms:/tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260924 21:13:45.415086 6986 mini_ranger_kms.cc:240] Using host: 127.6.210.148
I20260924 21:13:45.418773 6986 mini_ranger_kms.cc:276] Ranger KMS PID: 18101
I20260924 21:13:45.418916 6986 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.6.210.148:33161
Sep 24, 2026 9:13:46 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 24, 2026 9:13:46 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 24, 2026 9:13:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 24, 2026 9:13:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 24, 2026 9:13:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ranger-kms/ews/webapp].
Sep 24, 2026 9:13:46 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260924 21:13:47.731340 6986 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.191s user 0.018s sys 0.137s
I20260924 21:13:47.731520 6986 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260924 21:13:47.731632 6986 mini_ranger_kms.cc:401] 127.6.210.148:33161/kms/v1/keys
Sep 24 21:13:47 dist-test-slave-q276 krb5kdc[17708](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790284427, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.210.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 24 21:13:48 dist-test-slave-q276 krb5kdc[17708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284427, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.210.148@KRBTEST.COM for HTTP/127.6.210.148@KRBTEST.COM
Sep 24 21:13:48 dist-test-slave-q276 krb5kdc[17708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284427, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.210.148@KRBTEST.COM for HTTP/127.6.210.148@KRBTEST.COM
Sep 24 21:13:50 dist-test-slave-q276 krb5kdc[17708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.6.210.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
I20260924 21:13:51.001562 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:45369
--webserver_interface=127.6.210.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:38969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:45369
--ranger_config_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.210.148:33161/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:13:51.099123 18152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:13:51.099378 18152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:13:51.099419 18152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:13:51.100658 18152 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:13:51.100728 18152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:13:51.100754 18152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:13:51.100785 18152 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:13:51.100841 18152 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:13:51.102535 18152 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:38969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/master-0/wal
--ranger_kms_url=127.6.210.148:33161/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:45369
--ranger_config_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:45369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:13:51.102804 18152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:13:51.103029 18152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:13:51.108822 18157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:13:51.108925 18160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:13:51.108863 18152 server_base.cc:1034] running on GCE node
W20260924 21:13:51.108829 18158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:13:51.109481 18152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:13:51.109880 18152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:13:51.111044 18152 hybrid_clock.cc:648] HybridClock initialized: now 1790284431111019 us; error 38 us; skew 500 ppm
Sep 24 21:13:51 dist-test-slave-q276 krb5kdc[17708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284431, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:13:51.122488 18152 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:13:51.123097 18152 webserver.cc:460] Webserver started at http://127.6.210.190:42599/ using document root <none> and password file <none>
I20260924 21:13:51.123337 18152 fs_manager.cc:359] Metadata directory not provided
I20260924 21:13:51.123399 18152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:13:51.123551 18152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 21:13:51 dist-test-slave-q276 krb5kdc[17708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284431, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for HTTP/127.6.210.148@KRBTEST.COM
I20260924 21:13:51.185415 18152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "81d3656105f041f98ba7de268b9efab3"
format_stamp: "Formatted at 2026-09-24 21:13:51 on dist-test-slave-q276"
server_key: "3967c4954192fd5ba428c3e9c682757a"
server_key_iv: "3e3d4486fecdb51db8ae30014ada48d1"
server_key_version: "kuduclusterkey@0"
I20260924 21:13:51.186017 18152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "81d3656105f041f98ba7de268b9efab3"
format_stamp: "Formatted at 2026-09-24 21:13:51 on dist-test-slave-q276"
server_key: "3967c4954192fd5ba428c3e9c682757a"
server_key_iv: "3e3d4486fecdb51db8ae30014ada48d1"
server_key_version: "kuduclusterkey@0"
I20260924 21:13:51.187543 18152 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:13:51.203994 18167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:51.204685 18152 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260924 21:13:51.204888 18152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "81d3656105f041f98ba7de268b9efab3"
format_stamp: "Formatted at 2026-09-24 21:13:51 on dist-test-slave-q276"
server_key: "3967c4954192fd5ba428c3e9c682757a"
server_key_iv: "3e3d4486fecdb51db8ae30014ada48d1"
server_key_version: "kuduclusterkey@0"
I20260924 21:13:51.205006 18152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:13:51.223170 18152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:13:51.225324 18152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:13:51.225498 18152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:13:51.230561 18152 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:45369
I20260924 21:13:51.230577 18219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:45369 every 8 connection(s)
I20260924 21:13:51.230990 18152 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:13:51.231681 18220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:13:51.233950 18220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81d3656105f041f98ba7de268b9efab3: Bootstrap starting.
I20260924 21:13:51.234611 18220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81d3656105f041f98ba7de268b9efab3: Neither blocks nor log segments found. Creating new log.
I20260924 21:13:51.234910 18220 log.cc:824] T 00000000000000000000000000000000 P 81d3656105f041f98ba7de268b9efab3: Log is configured to *not* fsync() on all Append() calls
I20260924 21:13:51.235595 18220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81d3656105f041f98ba7de268b9efab3: No bootstrap required, opened a new log
I20260924 21:13:51.238068 18220 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81d3656105f041f98ba7de268b9efab3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81d3656105f041f98ba7de268b9efab3" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 45369 } }
I20260924 21:13:51.238216 18220 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81d3656105f041f98ba7de268b9efab3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:13:51.238238 18220 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81d3656105f041f98ba7de268b9efab3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81d3656105f041f98ba7de268b9efab3, State: Initialized, Role: FOLLOWER
I20260924 21:13:51.238473 18220 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81d3656105f041f98ba7de268b9efab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81d3656105f041f98ba7de268b9efab3" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 45369 } }
I20260924 21:13:51.238543 18220 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81d3656105f041f98ba7de268b9efab3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:13:51.238590 18220 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81d3656105f041f98ba7de268b9efab3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:13:51.238663 18220 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81d3656105f041f98ba7de268b9efab3 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:13:51.238751 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 18152
I20260924 21:13:51.238853 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:13:51.239033 6986 external_mini_cluster.cc:1428] Setting key 134deebf6bb8d7718e02e9c3eca85f50
I20260924 21:13:51.239470 18220 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81d3656105f041f98ba7de268b9efab3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81d3656105f041f98ba7de268b9efab3" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 45369 } }
I20260924 21:13:51.239609 18220 leader_election.cc:302] T 00000000000000000000000000000000 P 81d3656105f041f98ba7de268b9efab3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81d3656105f041f98ba7de268b9efab3; no voters:
I20260924 21:13:51.239794 18220 leader_election.cc:288] T 00000000000000000000000000000000 P 81d3656105f041f98ba7de268b9efab3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:13:51.239982 18226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81d3656105f041f98ba7de268b9efab3 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:13:51.240068 18220 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81d3656105f041f98ba7de268b9efab3 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:13:51.242167 18226 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81d3656105f041f98ba7de268b9efab3 [term 1 LEADER]: Becoming Leader. State: Replica: 81d3656105f041f98ba7de268b9efab3, State: Running, Role: LEADER
I20260924 21:13:51.242377 18226 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81d3656105f041f98ba7de268b9efab3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81d3656105f041f98ba7de268b9efab3" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 45369 } }
I20260924 21:13:51.243208 18220 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260924 21:13:51.245277 18237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81d3656105f041f98ba7de268b9efab3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81d3656105f041f98ba7de268b9efab3. Latest consensus state: current_term: 1 leader_uuid: "81d3656105f041f98ba7de268b9efab3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81d3656105f041f98ba7de268b9efab3" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 45369 } } }
I20260924 21:13:51.245573 18237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81d3656105f041f98ba7de268b9efab3 [sys.catalog]: This master's current role is: LEADER
I20260924 21:13:51.246076 18225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81d3656105f041f98ba7de268b9efab3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81d3656105f041f98ba7de268b9efab3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81d3656105f041f98ba7de268b9efab3" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 45369 } } }
I20260924 21:13:51.246142 18243 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:13:51.246217 18225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81d3656105f041f98ba7de268b9efab3 [sys.catalog]: This master's current role is: LEADER
I20260924 21:13:51.247396 18243 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 24 21:13:51 dist-test-slave-q276 krb5kdc[17708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:13:51.251730 18243 catalog_manager.cc:1347] Generated new cluster ID: 73af65ac03224f98927742a3612a303a
I20260924 21:13:51.251782 18243 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:13:51.255501 18224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:51.239576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33498 (local address 127.6.210.190:45369)
0924 21:13:51.239769 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:51.239781 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:51.239818 (+ 37us) server_negotiation.cc:407] Connection header received
0924 21:13:51.239949 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:51.239956 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:51.240016 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:51.240169 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:51.240894 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:51.242036 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:51.244848 (+ 2812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:51.246198 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:51.246390 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:51.249056 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:13:51.249060 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:13:51.249061 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:13:51.249077 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:13:51.251178 (+ 2101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:13:51.251570 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:13:51.251572 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:13:51.251573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:13:51.251689 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:13:51.254566 (+ 2877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:13:51.254567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:13:51.254567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:13:51.254827 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:13:51.254999 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0924 21:13:51.255124 (+ 125us) server_negotiation.cc:299] Negotiation successful
0924 21:13:51.255262 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":87,"threads_started":1}
I20260924 21:13:51.263746 18243 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:13:51.264590 18243 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:13:51.272928 18243 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81d3656105f041f98ba7de268b9efab3: Generated new TSK 0
I20260924 21:13:51.273241 18243 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.210.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
I20260924 21:13:52.014214 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.210.148:33161/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.129:0
--local_ip_for_outbound_sockets=127.6.210.129
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:45369
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:38969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:13:52.139263 18266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:13:52.139557 18266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:13:52.139601 18266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:13:52.141481 18266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:13:52.141541 18266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:13:52.141582 18266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.129
I20260924 21:13:52.143728 18266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:38969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.6.210.148:33161/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:45369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.129
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:13:52.144012 18266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:13:52.144312 18266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:13:52.151198 18274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:13:52.151310 18271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:13:52.151183 18266 server_base.cc:1034] running on GCE node
W20260924 21:13:52.151166 18272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:13:52.151752 18266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:13:52.152052 18266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:13:52.153229 18266 hybrid_clock.cc:648] HybridClock initialized: now 1790284432153213 us; error 35 us; skew 500 ppm
Sep 24 21:13:52 dist-test-slave-q276 krb5kdc[17708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284432, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:13:52.156517 18266 init.cc:377] Logged in from keytab as kudu/127.6.210.129@KRBTEST.COM (short username kudu)
I20260924 21:13:52.157121 18266 webserver.cc:460] Webserver started at http://127.6.210.129:38055/ using document root <none> and password file <none>
I20260924 21:13:52.157460 18266 fs_manager.cc:359] Metadata directory not provided
I20260924 21:13:52.157585 18266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:13:52.157768 18266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 21:13:52 dist-test-slave-q276 krb5kdc[17708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284432, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for HTTP/127.6.210.148@KRBTEST.COM
I20260924 21:13:52.171584 18266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-0/data/instance:
uuid: "ad4b13f27e8f4164ab698477c0742c86"
format_stamp: "Formatted at 2026-09-24 21:13:52 on dist-test-slave-q276"
server_key: "012576606d0ad3dc85bd29e85b1f59a6"
server_key_iv: "69933c5d498cd275940f21fc2cb2affe"
server_key_version: "kuduclusterkey@0"
I20260924 21:13:52.172060 18266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance:
uuid: "ad4b13f27e8f4164ab698477c0742c86"
format_stamp: "Formatted at 2026-09-24 21:13:52 on dist-test-slave-q276"
server_key: "012576606d0ad3dc85bd29e85b1f59a6"
server_key_iv: "69933c5d498cd275940f21fc2cb2affe"
server_key_version: "kuduclusterkey@0"
I20260924 21:13:52.173877 18266 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260924 21:13:52.184932 18281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:52.185165 18266 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:13:52.185277 18266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-0/wal
uuid: "ad4b13f27e8f4164ab698477c0742c86"
format_stamp: "Formatted at 2026-09-24 21:13:52 on dist-test-slave-q276"
server_key: "012576606d0ad3dc85bd29e85b1f59a6"
server_key_iv: "69933c5d498cd275940f21fc2cb2affe"
server_key_version: "kuduclusterkey@0"
I20260924 21:13:52.185400 18266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:13:52.194608 18266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:13:52.196628 18266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:13:52.196787 18266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:13:52.197043 18266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:13:52.197439 18266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:13:52.197496 18266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:52.197562 18266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:13:52.197605 18266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:52.205240 18266 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.129:40495
I20260924 21:13:52.205785 18266 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
I20260924 21:13:52.205983 18394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.129:40495 every 8 connection(s)
I20260924 21:13:52.211005 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 18266
I20260924 21:13:52.211117 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance
I20260924 21:13:52.211329 6986 external_mini_cluster.cc:1428] Setting key 2b0f5c4a4720f9f6af9703c27135738c
Sep 24 21:13:52 dist-test-slave-q276 krb5kdc[17708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284432, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:13:52.222632 18397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:52.207600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:45369 (local address 127.6.210.129:32957)
0924 21:13:52.208451 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:52.208485 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:52.209050 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:52.210591 (+ 1541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:52.210596 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:52.210790 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:13:52.211103 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:52.211110 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.212095 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.212096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:52.213446 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:52.213450 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.214033 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.214034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:52.214186 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:52.214842 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:13:52.214851 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:13:52.216782 (+ 1931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:13:52.218929 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:13:52.218936 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:13:52.218938 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:13:52.220151 (+ 1213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:13:52.221478 (+ 1327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:13:52.221478 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:13:52.221479 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:13:52.221622 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:13:52.221908 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:13:52.221913 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:13:52.222092 (+ 179us) client_negotiation.cc:769] Sending connection context
0924 21:13:52.222201 (+ 109us) client_negotiation.cc:240] Negotiation successful
0924 21:13:52.222391 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":807,"thread_start_us":81,"threads_started":1}
I20260924 21:13:52.222962 18398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:52.207612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:32957 (local address 127.6.210.190:45369)
0924 21:13:52.210378 (+ 2766us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:52.210379 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:52.210387 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:13:52.210410 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:52.210411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:52.210421 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:52.210506 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:52.211224 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.211997 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.213554 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.213901 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.213965 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:52.216894 (+ 2929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:13:52.216896 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:13:52.216897 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:13:52.216904 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:13:52.218840 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:13:52.221334 (+ 2494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:13:52.221335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:13:52.221336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:13:52.221407 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:13:52.221689 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:13:52.221690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:13:52.221690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:13:52.221828 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:13:52.221885 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 21:13:52.222778 (+ 893us) server_negotiation.cc:299] Negotiation successful
0924 21:13:52.222865 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2737,"thread_start_us":97,"threads_started":1}
I20260924 21:13:52.223527 18395 heartbeater.cc:347] Connected to a master server at 127.6.210.190:45369
I20260924 21:13:52.223634 18395 heartbeater.cc:464] Registering TS with master...
I20260924 21:13:52.223821 18395 heartbeater.cc:511] Master 127.6.210.190:45369 requested a full tablet report, sending...
I20260924 21:13:52.224442 18184 ts_manager.cc:194] Registered new tserver with Master: ad4b13f27e8f4164ab698477c0742c86 (127.6.210.129:40495)
WARNING: no policy specified for kudu/127.6.210.130@KRBTEST.COM; defaulting to no policy
I20260924 21:13:52.225330 18184 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.129@KRBTEST.COM'} at 127.6.210.129:32957
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
I20260924 21:13:52.245751 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.210.148:33161/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.130:0
--local_ip_for_outbound_sockets=127.6.210.130
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:45369
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:38969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:13:52.370545 18403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:13:52.370916 18403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:13:52.371003 18403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:13:52.372900 18403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:13:52.373019 18403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:13:52.373106 18403 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.130
I20260924 21:13:52.375466 18403 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:38969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.6.210.148:33161/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:45369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.130
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:13:52.375954 18403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:13:52.376279 18403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:13:52.396165 18410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:13:52.396178 18411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:13:52.396274 18413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:13:52.406229 18403 server_base.cc:1034] running on GCE node
I20260924 21:13:52.406543 18403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:13:52.406855 18403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:13:52.408004 18403 hybrid_clock.cc:648] HybridClock initialized: now 1790284432407968 us; error 53 us; skew 500 ppm
Sep 24 21:13:52 dist-test-slave-q276 krb5kdc[17708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284432, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:13:52.411763 18403 init.cc:377] Logged in from keytab as kudu/127.6.210.130@KRBTEST.COM (short username kudu)
I20260924 21:13:52.412339 18403 webserver.cc:460] Webserver started at http://127.6.210.130:44523/ using document root <none> and password file <none>
I20260924 21:13:52.412631 18403 fs_manager.cc:359] Metadata directory not provided
I20260924 21:13:52.412691 18403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:13:52.412830 18403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 21:13:52 dist-test-slave-q276 krb5kdc[17708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284432, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for HTTP/127.6.210.148@KRBTEST.COM
I20260924 21:13:52.425907 18403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-1/data/instance:
uuid: "6609fb483edd43bd8667c5cf225de678"
format_stamp: "Formatted at 2026-09-24 21:13:52 on dist-test-slave-q276"
server_key: "6ed30e4afb29abc8ac567d5540d28a5c"
server_key_iv: "68c740d8909c7cccdf35aa398d5ea11e"
server_key_version: "kuduclusterkey@0"
I20260924 21:13:52.428802 18403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance:
uuid: "6609fb483edd43bd8667c5cf225de678"
format_stamp: "Formatted at 2026-09-24 21:13:52 on dist-test-slave-q276"
server_key: "6ed30e4afb29abc8ac567d5540d28a5c"
server_key_iv: "68c740d8909c7cccdf35aa398d5ea11e"
server_key_version: "kuduclusterkey@0"
I20260924 21:13:52.430985 18403 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260924 21:13:52.452292 18420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:52.452509 18403 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260924 21:13:52.452606 18403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-1/wal
uuid: "6609fb483edd43bd8667c5cf225de678"
format_stamp: "Formatted at 2026-09-24 21:13:52 on dist-test-slave-q276"
server_key: "6ed30e4afb29abc8ac567d5540d28a5c"
server_key_iv: "68c740d8909c7cccdf35aa398d5ea11e"
server_key_version: "kuduclusterkey@0"
I20260924 21:13:52.452718 18403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:13:52.468240 18403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
Sep 24 21:13:52 dist-test-slave-q276 krb5kdc[17708](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790284432, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:13:52.470902 18403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:13:52.471098 18403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:13:52.471387 18403 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:13:52.471808 18403 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:13:52.471868 18403 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:52.471932 18403 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:13:52.471964 18403 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:52.479863 18403 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.130:46381
I20260924 21:13:52.480271 18403 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
I20260924 21:13:52.481606 18533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.130:46381 every 8 connection(s)
I20260924 21:13:52.486375 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 18403
I20260924 21:13:52.486497 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance
I20260924 21:13:52.486718 6986 external_mini_cluster.cc:1428] Setting key 44f92460d10381e2867c577f6af8a076
Sep 24 21:13:52 dist-test-slave-q276 krb5kdc[17708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284432, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:13:52.495615 18398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:52.481914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:35225 (local address 127.6.210.190:45369)
0924 21:13:52.481968 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:52.481971 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:52.482161 (+ 190us) server_negotiation.cc:407] Connection header received
0924 21:13:52.482824 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:52.482825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:52.482840 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:52.482903 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:52.483527 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.484091 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.485433 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.485795 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.485837 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:52.491710 (+ 5873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:13:52.491712 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:13:52.491712 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:13:52.491721 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:13:52.493655 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:13:52.494242 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:13:52.494244 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:13:52.494244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:13:52.494373 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:13:52.494716 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:13:52.494717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:13:52.494718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:13:52.494901 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:13:52.494951 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 21:13:52.495385 (+ 434us) server_negotiation.cc:299] Negotiation successful
0924 21:13:52.495487 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 21:13:52.496680 18536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:52.481613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:45369 (local address 127.6.210.130:35225)
0924 21:13:52.482004 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:52.482052 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:52.482694 (+ 642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:52.482986 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:52.482990 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:52.483201 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:13:52.483406 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:52.483411 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.484200 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.484202 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:52.485314 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:52.485318 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.485950 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.485951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:52.486102 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:52.489812 (+ 3710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:13:52.489822 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:13:52.491574 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:13:52.493768 (+ 2194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:13:52.493775 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:13:52.493778 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:13:52.494138 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:13:52.494462 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:13:52.494463 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:13:52.494463 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:13:52.494625 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:13:52.495007 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:13:52.495013 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:13:52.495197 (+ 184us) client_negotiation.cc:769] Sending connection context
0924 21:13:52.496193 (+ 996us) client_negotiation.cc:240] Negotiation successful
0924 21:13:52.496402 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":84,"threads_started":1}
I20260924 21:13:52.497412 18534 heartbeater.cc:347] Connected to a master server at 127.6.210.190:45369
I20260924 21:13:52.497527 18534 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.6.210.131@KRBTEST.COM; defaulting to no policy
I20260924 21:13:52.497733 18534 heartbeater.cc:511] Master 127.6.210.190:45369 requested a full tablet report, sending...
I20260924 21:13:52.498226 18184 ts_manager.cc:194] Registered new tserver with Master: 6609fb483edd43bd8667c5cf225de678 (127.6.210.130:46381)
I20260924 21:13:52.498837 18184 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.130@KRBTEST.COM'} at 127.6.210.130:35225
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
I20260924 21:13:52.534113 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.210.148:33161/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.131:0
--local_ip_for_outbound_sockets=127.6.210.131
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:45369
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:38969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:13:52.649089 18539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:13:52.649461 18539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:13:52.649549 18539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:13:52.651388 18539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:13:52.651461 18539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:13:52.651501 18539 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.131
I20260924 21:13:52.653983 18539 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:38969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.6.210.148:33161/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:45369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.131
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:13:52.654299 18539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:13:52.654618 18539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:13:52.662254 18548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:13:52.662361 18546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:13:52.662750 18539 server_base.cc:1034] running on GCE node
W20260924 21:13:52.662616 18545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:13:52.662985 18539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:13:52.663276 18539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:13:52.664455 18539 hybrid_clock.cc:648] HybridClock initialized: now 1790284432664427 us; error 44 us; skew 500 ppm
Sep 24 21:13:52 dist-test-slave-q276 krb5kdc[17708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284432, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:13:52.667841 18539 init.cc:377] Logged in from keytab as kudu/127.6.210.131@KRBTEST.COM (short username kudu)
I20260924 21:13:52.668464 18539 webserver.cc:460] Webserver started at http://127.6.210.131:36955/ using document root <none> and password file <none>
I20260924 21:13:52.668767 18539 fs_manager.cc:359] Metadata directory not provided
I20260924 21:13:52.668848 18539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:13:52.668999 18539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 21:13:52 dist-test-slave-q276 krb5kdc[17708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284432, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for HTTP/127.6.210.148@KRBTEST.COM
I20260924 21:13:52.683811 18539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-2/data/instance:
uuid: "f211c129006447348227929f7de24c3c"
format_stamp: "Formatted at 2026-09-24 21:13:52 on dist-test-slave-q276"
server_key: "1a4dc4f975da25dc121f2fade973e0fc"
server_key_iv: "6722edb5cca072a8e44bfd6fbc1eb5c2"
server_key_version: "kuduclusterkey@0"
I20260924 21:13:52.684383 18539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance:
uuid: "f211c129006447348227929f7de24c3c"
format_stamp: "Formatted at 2026-09-24 21:13:52 on dist-test-slave-q276"
server_key: "1a4dc4f975da25dc121f2fade973e0fc"
server_key_iv: "6722edb5cca072a8e44bfd6fbc1eb5c2"
server_key_version: "kuduclusterkey@0"
I20260924 21:13:52.690650 18539 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.000s
I20260924 21:13:52.706410 18555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:52.706669 18539 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260924 21:13:52.706758 18539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-2/wal
uuid: "f211c129006447348227929f7de24c3c"
format_stamp: "Formatted at 2026-09-24 21:13:52 on dist-test-slave-q276"
server_key: "1a4dc4f975da25dc121f2fade973e0fc"
server_key_iv: "6722edb5cca072a8e44bfd6fbc1eb5c2"
server_key_version: "kuduclusterkey@0"
I20260924 21:13:52.706848 18539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:13:52.732673 18539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:13:52.735213 18539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:13:52.735433 18539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:13:52.735730 18539 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:13:52.736155 18539 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:13:52.736251 18539 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:52.736330 18539 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:13:52.736389 18539 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:13:52.743391 18539 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.131:46665
I20260924 21:13:52.743856 18539 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
I20260924 21:13:52.750445 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 18539
I20260924 21:13:52.750547 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance
I20260924 21:13:52.750722 6986 external_mini_cluster.cc:1428] Setting key 3067eed35ff00ff638350587c359cad6
I20260924 21:13:52.754509 18668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.131:46665 every 8 connection(s)
Sep 24 21:13:52 dist-test-slave-q276 krb5kdc[17708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284432, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:13:52.762670 18398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:52.745656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:38365 (local address 127.6.210.190:45369)
0924 21:13:52.745702 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:52.745704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:52.745906 (+ 202us) server_negotiation.cc:407] Connection header received
0924 21:13:52.746726 (+ 820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:52.746728 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:52.746741 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:52.746773 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:52.748028 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.748800 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.755850 (+ 7050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.756188 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.756279 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:52.759422 (+ 3143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:13:52.759424 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:13:52.759424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:13:52.759431 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:13:52.760967 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:13:52.761444 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:13:52.761444 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:13:52.761445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:13:52.761494 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:13:52.761770 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:13:52.761771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:13:52.761771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:13:52.761912 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:13:52.761998 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0924 21:13:52.762493 (+ 495us) server_negotiation.cc:299] Negotiation successful
0924 21:13:52.762583 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 21:13:52.762921 18671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:52.744888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:45369 (local address 127.6.210.131:38365)
0924 21:13:52.745776 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:52.745822 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:52.746593 (+ 771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:52.747377 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:52.747381 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:52.747630 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:13:52.747913 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:52.747920 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.754423 (+ 6503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.754424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:52.755728 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:52.755732 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.756666 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.756667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:52.756830 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:52.757553 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:13:52.757564 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:13:52.759241 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:13:52.761050 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:13:52.761057 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:13:52.761059 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:13:52.761366 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:13:52.761560 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:13:52.761561 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:13:52.761561 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:13:52.761707 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:13:52.761988 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:13:52.761994 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:13:52.762201 (+ 207us) client_negotiation.cc:769] Sending connection context
0924 21:13:52.762417 (+ 216us) client_negotiation.cc:240] Negotiation successful
0924 21:13:52.762620 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":845,"thread_start_us":561,"threads_started":1}
I20260924 21:13:52.763388 18669 heartbeater.cc:347] Connected to a master server at 127.6.210.190:45369
I20260924 21:13:52.763502 18669 heartbeater.cc:464] Registering TS with master...
I20260924 21:13:52.763712 18669 heartbeater.cc:511] Master 127.6.210.190:45369 requested a full tablet report, sending...
I20260924 21:13:52.764119 18184 ts_manager.cc:194] Registered new tserver with Master: f211c129006447348227929f7de24c3c (127.6.210.131:46665)
I20260924 21:13:52.764504 18184 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.131@KRBTEST.COM'} at 127.6.210.131:38365
I20260924 21:13:52.773597 6986 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:13:52.783391 18398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:52.776010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33514 (local address 127.6.210.190:45369)
0924 21:13:52.776082 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:52.776084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:52.776155 (+ 71us) server_negotiation.cc:407] Connection header received
0924 21:13:52.776210 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:52.776211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:52.776224 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:52.776296 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:13:52.776882 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.777607 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.778706 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.778994 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.779032 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:52.780595 (+ 1563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:13:52.780596 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:13:52.780597 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:13:52.780604 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:13:52.782205 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:13:52.782599 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:13:52.782600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:13:52.782600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:13:52.782653 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:13:52.782838 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:13:52.782838 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:13:52.782839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:13:52.782952 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:13:52.783021 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 21:13:52.783226 (+ 205us) server_negotiation.cc:299] Negotiation successful
0924 21:13:52.783299 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 21:13:52.785586 18184 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33514:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:13:52.787767 18184 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:13:52.801779 18684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:52.794254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:46665 (local address 127.0.0.1:36660)
0924 21:13:52.796156 (+ 1902us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:52.796165 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:52.796211 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:52.796452 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:52.796453 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:52.796455 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:13:52.796493 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:52.796494 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.797824 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.797825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:52.799169 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:52.799173 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.801458 (+ 2285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.801459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:52.801541 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:52.801555 (+ 14us) client_negotiation.cc:769] Sending connection context
0924 21:13:52.801642 (+ 87us) client_negotiation.cc:240] Negotiation successful
0924 21:13:52.801693 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1886,"thread_start_us":620,"threads_started":1}
I20260924 21:13:52.802433 18683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:52.794684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36660 (local address 127.6.210.131:46665)
0924 21:13:52.796104 (+ 1420us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:52.796110 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:52.796264 (+ 154us) server_negotiation.cc:407] Connection header received
0924 21:13:52.796302 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:52.796304 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:52.796319 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:52.796383 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:13:52.796564 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.797724 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.800520 (+ 2796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.801272 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.801345 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:52.801420 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 21:13:52.802305 (+ 885us) server_negotiation.cc:299] Negotiation successful
0924 21:13:52.802351 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1385,"thread_start_us":85,"threads_started":1}
I20260924 21:13:52.802627 18685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:52.794248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:46381 (local address 127.0.0.1:48898)
0924 21:13:52.795317 (+ 1069us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:52.795358 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:52.795427 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:52.795891 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:52.795894 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:52.795896 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:13:52.795970 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:52.795972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.799269 (+ 3297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.799270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:52.800401 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:52.800404 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.802402 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.802403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:52.802469 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:52.802473 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:13:52.802519 (+ 46us) client_negotiation.cc:240] Negotiation successful
0924 21:13:52.802556 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1034,"mutex_wait_us":1,"thread_start_us":100,"threads_started":1}
I20260924 21:13:52.803103 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:52.794495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48898 (local address 127.6.210.130:46381)
0924 21:13:52.795654 (+ 1159us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:52.795658 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:52.795668 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:13:52.795707 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:52.795710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:52.795726 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:52.795802 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:13:52.797392 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.798522 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.801437 (+ 2915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.802219 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.802301 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:52.802370 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 21:13:52.802966 (+ 596us) server_negotiation.cc:299] Negotiation successful
0924 21:13:52.803008 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1127,"thread_start_us":87,"threads_started":1}
I20260924 21:13:52.803325 18603 tablet_service.cc:1467] Processing CreateTablet for tablet 5509924433b144f1be541f98ebb8ca97 (DEFAULT_TABLE table=test-table [id=6ee1811d1eb340348ad300026bde0a01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:13:52.803714 18603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5509924433b144f1be541f98ebb8ca97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:13:52.804579 18681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:52.794986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:40495 (local address 127.0.0.1:34784)
0924 21:13:52.798659 (+ 3673us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:52.798670 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:52.798715 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:52.798958 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:52.798959 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:52.798961 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:13:52.799000 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:52.799002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.800227 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.800228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:52.803360 (+ 3132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:52.803363 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.804358 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.804359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:52.804423 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:52.804426 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:13:52.804472 (+ 46us) client_negotiation.cc:240] Negotiation successful
0924 21:13:52.804511 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3648,"thread_start_us":56,"threads_started":1}
I20260924 21:13:52.804843 18686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:52.795170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34784 (local address 127.6.210.129:40495)
0924 21:13:52.795560 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:52.795565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:52.798773 (+ 3208us) server_negotiation.cc:407] Connection header received
0924 21:13:52.798811 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:52.798814 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:52.798829 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:52.798892 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:13:52.799082 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.800129 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.803455 (+ 3326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.804206 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.804277 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:52.804325 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 21:13:52.804721 (+ 396us) server_negotiation.cc:299] Negotiation successful
0924 21:13:52.804767 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":55,"threads_started":1}
I20260924 21:13:52.805420 18329 tablet_service.cc:1467] Processing CreateTablet for tablet 5509924433b144f1be541f98ebb8ca97 (DEFAULT_TABLE table=test-table [id=6ee1811d1eb340348ad300026bde0a01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:13:52.805732 18329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5509924433b144f1be541f98ebb8ca97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:13:52.808068 18468 tablet_service.cc:1467] Processing CreateTablet for tablet 5509924433b144f1be541f98ebb8ca97 (DEFAULT_TABLE table=test-table [id=6ee1811d1eb340348ad300026bde0a01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:13:52.808386 18468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5509924433b144f1be541f98ebb8ca97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:13:52.811205 18687 tablet_bootstrap.cc:492] T 5509924433b144f1be541f98ebb8ca97 P ad4b13f27e8f4164ab698477c0742c86: Bootstrap starting.
I20260924 21:13:52.811992 18687 tablet_bootstrap.cc:654] T 5509924433b144f1be541f98ebb8ca97 P ad4b13f27e8f4164ab698477c0742c86: Neither blocks nor log segments found. Creating new log.
I20260924 21:13:52.812392 18687 log.cc:824] T 5509924433b144f1be541f98ebb8ca97 P ad4b13f27e8f4164ab698477c0742c86: Log is configured to *not* fsync() on all Append() calls
I20260924 21:13:52.813189 18687 tablet_bootstrap.cc:492] T 5509924433b144f1be541f98ebb8ca97 P ad4b13f27e8f4164ab698477c0742c86: No bootstrap required, opened a new log
I20260924 21:13:52.813254 18687 ts_tablet_manager.cc:1397] T 5509924433b144f1be541f98ebb8ca97 P ad4b13f27e8f4164ab698477c0742c86: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:13:52.815927 18687 raft_consensus.cc:348] T 5509924433b144f1be541f98ebb8ca97 P ad4b13f27e8f4164ab698477c0742c86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f211c129006447348227929f7de24c3c" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46665 } } peers { permanent_uuid: "ad4b13f27e8f4164ab698477c0742c86" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 40495 } } peers { permanent_uuid: "6609fb483edd43bd8667c5cf225de678" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 46381 } }
I20260924 21:13:52.816110 18687 raft_consensus.cc:374] T 5509924433b144f1be541f98ebb8ca97 P ad4b13f27e8f4164ab698477c0742c86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:13:52.816145 18687 raft_consensus.cc:729] T 5509924433b144f1be541f98ebb8ca97 P ad4b13f27e8f4164ab698477c0742c86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad4b13f27e8f4164ab698477c0742c86, State: Initialized, Role: FOLLOWER
I20260924 21:13:52.816799 18687 consensus_queue.cc:260] T 5509924433b144f1be541f98ebb8ca97 P ad4b13f27e8f4164ab698477c0742c86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f211c129006447348227929f7de24c3c" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46665 } } peers { permanent_uuid: "ad4b13f27e8f4164ab698477c0742c86" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 40495 } } peers { permanent_uuid: "6609fb483edd43bd8667c5cf225de678" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 46381 } }
I20260924 21:13:52.817132 18687 ts_tablet_manager.cc:1428] T 5509924433b144f1be541f98ebb8ca97 P ad4b13f27e8f4164ab698477c0742c86: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260924 21:13:52.817322 18395 heartbeater.cc:503] Master 127.6.210.190:45369 was elected leader, sending a full tablet report...
I20260924 21:13:52.821461 18691 tablet_bootstrap.cc:492] T 5509924433b144f1be541f98ebb8ca97 P 6609fb483edd43bd8667c5cf225de678: Bootstrap starting.
I20260924 21:13:52.822834 18691 tablet_bootstrap.cc:654] T 5509924433b144f1be541f98ebb8ca97 P 6609fb483edd43bd8667c5cf225de678: Neither blocks nor log segments found. Creating new log.
I20260924 21:13:52.823071 18690 tablet_bootstrap.cc:492] T 5509924433b144f1be541f98ebb8ca97 P f211c129006447348227929f7de24c3c: Bootstrap starting.
I20260924 21:13:52.824301 18690 tablet_bootstrap.cc:654] T 5509924433b144f1be541f98ebb8ca97 P f211c129006447348227929f7de24c3c: Neither blocks nor log segments found. Creating new log.
I20260924 21:13:52.824627 18690 log.cc:824] T 5509924433b144f1be541f98ebb8ca97 P f211c129006447348227929f7de24c3c: Log is configured to *not* fsync() on all Append() calls
I20260924 21:13:52.825233 18691 log.cc:824] T 5509924433b144f1be541f98ebb8ca97 P 6609fb483edd43bd8667c5cf225de678: Log is configured to *not* fsync() on all Append() calls
I20260924 21:13:52.825789 18690 tablet_bootstrap.cc:492] T 5509924433b144f1be541f98ebb8ca97 P f211c129006447348227929f7de24c3c: No bootstrap required, opened a new log
I20260924 21:13:52.825857 18690 ts_tablet_manager.cc:1397] T 5509924433b144f1be541f98ebb8ca97 P f211c129006447348227929f7de24c3c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260924 21:13:52.834182 18690 raft_consensus.cc:348] T 5509924433b144f1be541f98ebb8ca97 P f211c129006447348227929f7de24c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f211c129006447348227929f7de24c3c" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46665 } } peers { permanent_uuid: "ad4b13f27e8f4164ab698477c0742c86" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 40495 } } peers { permanent_uuid: "6609fb483edd43bd8667c5cf225de678" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 46381 } }
I20260924 21:13:52.834440 18690 raft_consensus.cc:374] T 5509924433b144f1be541f98ebb8ca97 P f211c129006447348227929f7de24c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:13:52.834486 18690 raft_consensus.cc:729] T 5509924433b144f1be541f98ebb8ca97 P f211c129006447348227929f7de24c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f211c129006447348227929f7de24c3c, State: Initialized, Role: FOLLOWER
I20260924 21:13:52.835049 18691 tablet_bootstrap.cc:492] T 5509924433b144f1be541f98ebb8ca97 P 6609fb483edd43bd8667c5cf225de678: No bootstrap required, opened a new log
I20260924 21:13:52.835155 18691 ts_tablet_manager.cc:1397] T 5509924433b144f1be541f98ebb8ca97 P 6609fb483edd43bd8667c5cf225de678: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.000s
I20260924 21:13:52.835563 18690 consensus_queue.cc:260] T 5509924433b144f1be541f98ebb8ca97 P f211c129006447348227929f7de24c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f211c129006447348227929f7de24c3c" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46665 } } peers { permanent_uuid: "ad4b13f27e8f4164ab698477c0742c86" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 40495 } } peers { permanent_uuid: "6609fb483edd43bd8667c5cf225de678" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 46381 } }
I20260924 21:13:52.835945 18690 ts_tablet_manager.cc:1428] T 5509924433b144f1be541f98ebb8ca97 P f211c129006447348227929f7de24c3c: Time spent starting tablet: real 0.010s user 0.003s sys 0.000s
I20260924 21:13:52.836118 18669 heartbeater.cc:503] Master 127.6.210.190:45369 was elected leader, sending a full tablet report...
I20260924 21:13:52.837289 18691 raft_consensus.cc:348] T 5509924433b144f1be541f98ebb8ca97 P 6609fb483edd43bd8667c5cf225de678 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f211c129006447348227929f7de24c3c" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46665 } } peers { permanent_uuid: "ad4b13f27e8f4164ab698477c0742c86" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 40495 } } peers { permanent_uuid: "6609fb483edd43bd8667c5cf225de678" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 46381 } }
I20260924 21:13:52.837450 18691 raft_consensus.cc:374] T 5509924433b144f1be541f98ebb8ca97 P 6609fb483edd43bd8667c5cf225de678 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:13:52.837486 18691 raft_consensus.cc:729] T 5509924433b144f1be541f98ebb8ca97 P 6609fb483edd43bd8667c5cf225de678 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6609fb483edd43bd8667c5cf225de678, State: Initialized, Role: FOLLOWER
I20260924 21:13:52.837605 18691 consensus_queue.cc:260] T 5509924433b144f1be541f98ebb8ca97 P 6609fb483edd43bd8667c5cf225de678 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f211c129006447348227929f7de24c3c" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46665 } } peers { permanent_uuid: "ad4b13f27e8f4164ab698477c0742c86" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 40495 } } peers { permanent_uuid: "6609fb483edd43bd8667c5cf225de678" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 46381 } }
I20260924 21:13:52.837862 18691 ts_tablet_manager.cc:1428] T 5509924433b144f1be541f98ebb8ca97 P 6609fb483edd43bd8667c5cf225de678: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:13:52.838037 18534 heartbeater.cc:503] Master 127.6.210.190:45369 was elected leader, sending a full tablet report...
I20260924 21:13:52.875036 18694 raft_consensus.cc:482] T 5509924433b144f1be541f98ebb8ca97 P f211c129006447348227929f7de24c3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:13:52.875187 18694 raft_consensus.cc:504] T 5509924433b144f1be541f98ebb8ca97 P f211c129006447348227929f7de24c3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f211c129006447348227929f7de24c3c" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46665 } } peers { permanent_uuid: "ad4b13f27e8f4164ab698477c0742c86" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 40495 } } peers { permanent_uuid: "6609fb483edd43bd8667c5cf225de678" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 46381 } }
I20260924 21:13:52.875600 18694 leader_election.cc:288] T 5509924433b144f1be541f98ebb8ca97 P f211c129006447348227929f7de24c3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad4b13f27e8f4164ab698477c0742c86 (127.6.210.129:40495), 6609fb483edd43bd8667c5cf225de678 (127.6.210.130:46381)
I20260924 21:13:52.880303 18671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:52.875886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:46381 (local address 127.6.210.131:60213)
0924 21:13:52.876107 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:52.876120 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:52.876169 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:52.876487 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:52.876488 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:52.876490 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:13:52.876558 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:52.876560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.877515 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.877516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:52.878743 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:52.878746 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.879973 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.879974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:52.880063 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:52.880068 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 21:13:52.880138 (+ 70us) client_negotiation.cc:240] Negotiation successful
0924 21:13:52.880230 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":37}
I20260924 21:13:52.880309 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:52.876262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:60213 (local address 127.6.210.130:46381)
0924 21:13:52.876315 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:52.876318 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:52.876330 (+ 12us) server_negotiation.cc:407] Connection header received
0924 21:13:52.876357 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:52.876358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:52.876368 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:52.876433 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:13:52.876630 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.877401 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.879339 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.879888 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.879964 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:52.879973 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 21:13:52.880193 (+ 220us) server_negotiation.cc:299] Negotiation successful
0924 21:13:52.880239 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260924 21:13:52.880734 18481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5509924433b144f1be541f98ebb8ca97" candidate_uuid: "f211c129006447348227929f7de24c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6609fb483edd43bd8667c5cf225de678" is_pre_election: true
I20260924 21:13:52.880904 18481 raft_consensus.cc:2454] T 5509924433b144f1be541f98ebb8ca97 P 6609fb483edd43bd8667c5cf225de678 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f211c129006447348227929f7de24c3c in term 0.
I20260924 21:13:52.881156 18557 leader_election.cc:302] T 5509924433b144f1be541f98ebb8ca97 P f211c129006447348227929f7de24c3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6609fb483edd43bd8667c5cf225de678, f211c129006447348227929f7de24c3c; no voters:
I20260924 21:13:52.881413 18694 raft_consensus.cc:2781] T 5509924433b144f1be541f98ebb8ca97 P f211c129006447348227929f7de24c3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:13:52.881470 18694 raft_consensus.cc:482] T 5509924433b144f1be541f98ebb8ca97 P f211c129006447348227929f7de24c3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:13:52.881495 18694 raft_consensus.cc:3037] T 5509924433b144f1be541f98ebb8ca97 P f211c129006447348227929f7de24c3c [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:13:52.882484 18686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:52.876619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:35579 (local address 127.6.210.129:40495)
0924 21:13:52.877536 (+ 917us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:52.877538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:52.878188 (+ 650us) server_negotiation.cc:407] Connection header received
0924 21:13:52.878319 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:52.878320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:52.878330 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:52.878358 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:13:52.878556 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.879263 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.881492 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.882084 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.882129 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:52.882136 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 21:13:52.882376 (+ 240us) server_negotiation.cc:299] Negotiation successful
0924 21:13:52.882418 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":890}
I20260924 21:13:52.882453 18694 raft_consensus.cc:504] T 5509924433b144f1be541f98ebb8ca97 P f211c129006447348227929f7de24c3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f211c129006447348227929f7de24c3c" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46665 } } peers { permanent_uuid: "ad4b13f27e8f4164ab698477c0742c86" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 40495 } } peers { permanent_uuid: "6609fb483edd43bd8667c5cf225de678" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 46381 } }
I20260924 21:13:52.882579 18694 leader_election.cc:288] T 5509924433b144f1be541f98ebb8ca97 P f211c129006447348227929f7de24c3c [CANDIDATE]: Term 1 election: Requested vote from peers ad4b13f27e8f4164ab698477c0742c86 (127.6.210.129:40495), 6609fb483edd43bd8667c5cf225de678 (127.6.210.130:46381)
I20260924 21:13:52.882679 18696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:52.876415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:40495 (local address 127.6.210.131:35579)
0924 21:13:52.878121 (+ 1706us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:52.878133 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:13:52.878214 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:13:52.878419 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:13:52.878420 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:13:52.878421 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:13:52.878491 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:52.878492 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.880030 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.880031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:52.881390 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:13:52.881392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.882212 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.882213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:13:52.882297 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:52.882301 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:13:52.882549 (+ 248us) client_negotiation.cc:240] Negotiation successful
0924 21:13:52.882588 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1688,"thread_start_us":86,"threads_started":1}
I20260924 21:13:52.883085 18348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5509924433b144f1be541f98ebb8ca97" candidate_uuid: "f211c129006447348227929f7de24c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad4b13f27e8f4164ab698477c0742c86"
I20260924 21:13:52.883203 18348 raft_consensus.cc:3037] T 5509924433b144f1be541f98ebb8ca97 P ad4b13f27e8f4164ab698477c0742c86 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:13:52.883231 18481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5509924433b144f1be541f98ebb8ca97" candidate_uuid: "f211c129006447348227929f7de24c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6609fb483edd43bd8667c5cf225de678"
I20260924 21:13:52.883308 18481 raft_consensus.cc:3037] T 5509924433b144f1be541f98ebb8ca97 P 6609fb483edd43bd8667c5cf225de678 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:13:52.884027 18348 raft_consensus.cc:2454] T 5509924433b144f1be541f98ebb8ca97 P ad4b13f27e8f4164ab698477c0742c86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f211c129006447348227929f7de24c3c in term 1.
I20260924 21:13:52.883082 18349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5509924433b144f1be541f98ebb8ca97" candidate_uuid: "f211c129006447348227929f7de24c3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad4b13f27e8f4164ab698477c0742c86" is_pre_election: true
I20260924 21:13:52.884169 18481 raft_consensus.cc:2454] T 5509924433b144f1be541f98ebb8ca97 P 6609fb483edd43bd8667c5cf225de678 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f211c129006447348227929f7de24c3c in term 1.
I20260924 21:13:52.884197 18349 raft_consensus.cc:2362] T 5509924433b144f1be541f98ebb8ca97 P ad4b13f27e8f4164ab698477c0742c86 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f211c129006447348227929f7de24c3c in term 1. Re-sending same reply.
I20260924 21:13:52.884438 18557 leader_election.cc:302] T 5509924433b144f1be541f98ebb8ca97 P f211c129006447348227929f7de24c3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6609fb483edd43bd8667c5cf225de678, f211c129006447348227929f7de24c3c; no voters:
I20260924 21:13:52.884568 18694 raft_consensus.cc:2781] T 5509924433b144f1be541f98ebb8ca97 P f211c129006447348227929f7de24c3c [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:13:52.884709 18694 raft_consensus.cc:686] T 5509924433b144f1be541f98ebb8ca97 P f211c129006447348227929f7de24c3c [term 1 LEADER]: Becoming Leader. State: Replica: f211c129006447348227929f7de24c3c, State: Running, Role: LEADER
I20260924 21:13:52.884796 18694 consensus_queue.cc:237] T 5509924433b144f1be541f98ebb8ca97 P f211c129006447348227929f7de24c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f211c129006447348227929f7de24c3c" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46665 } } peers { permanent_uuid: "ad4b13f27e8f4164ab698477c0742c86" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 40495 } } peers { permanent_uuid: "6609fb483edd43bd8667c5cf225de678" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 46381 } }
I20260924 21:13:52.885582 18184 catalog_manager.cc:5495] T 5509924433b144f1be541f98ebb8ca97 P f211c129006447348227929f7de24c3c reported cstate change: term changed from 0 to 1, leader changed from <none> to f211c129006447348227929f7de24c3c (127.6.210.131). New cstate: current_term: 1 leader_uuid: "f211c129006447348227929f7de24c3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f211c129006447348227929f7de24c3c" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46665 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad4b13f27e8f4164ab698477c0742c86" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 40495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6609fb483edd43bd8667c5cf225de678" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 46381 } health_report { overall_health: UNKNOWN } } }
I20260924 21:13:52.898780 18683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:52.895636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36664 (local address 127.6.210.131:46665)
0924 21:13:52.895676 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 21:13:52.895678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:13:52.895687 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:13:52.895748 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:13:52.895749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:13:52.895768 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:13:52.895852 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:13:52.895990 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.896727 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.897836 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:13:52.898141 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:13:52.898195 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:13:52.898416 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:13:52.898546 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:13:52.898624 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0924 21:13:52.898684 (+ 60us) server_negotiation.cc:299] Negotiation successful
0924 21:13:52.898705 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 21:13:52.901216 18349 raft_consensus.cc:1261] T 5509924433b144f1be541f98ebb8ca97 P ad4b13f27e8f4164ab698477c0742c86 [term 1 FOLLOWER]: Refusing update from remote peer f211c129006447348227929f7de24c3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:13:52.901643 18481 raft_consensus.cc:1261] T 5509924433b144f1be541f98ebb8ca97 P 6609fb483edd43bd8667c5cf225de678 [term 1 FOLLOWER]: Refusing update from remote peer f211c129006447348227929f7de24c3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:13:52.901623 18697 consensus_queue.cc:1035] T 5509924433b144f1be541f98ebb8ca97 P f211c129006447348227929f7de24c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad4b13f27e8f4164ab698477c0742c86" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 40495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:13:52.902043 18697 consensus_queue.cc:1035] T 5509924433b144f1be541f98ebb8ca97 P f211c129006447348227929f7de24c3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6609fb483edd43bd8667c5cf225de678" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 46381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:13:52.909387 18184 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:13:52.909497 18184 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:13:52.910223 18184 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 81d3656105f041f98ba7de268b9efab3: Sending DeleteTablet for 3 replicas of tablet 5509924433b144f1be541f98ebb8ca97
I20260924 21:13:52.910676 18603 tablet_service.cc:1514] Processing DeleteTablet for tablet 5509924433b144f1be541f98ebb8ca97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:13:52 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:36660
I20260924 21:13:52.910912 18329 tablet_service.cc:1514] Processing DeleteTablet for tablet 5509924433b144f1be541f98ebb8ca97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:13:52 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:34784
I20260924 21:13:52.911311 18468 tablet_service.cc:1514] Processing DeleteTablet for tablet 5509924433b144f1be541f98ebb8ca97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:13:52 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:48898
I20260924 21:13:52.911640 18707 tablet_replica.cc:330] T 5509924433b144f1be541f98ebb8ca97 P f211c129006447348227929f7de24c3c: stopping tablet replica
I20260924 21:13:52.911778 18707 raft_consensus.cc:2229] T 5509924433b144f1be541f98ebb8ca97 P f211c129006447348227929f7de24c3c [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:13:52.911886 18707 raft_consensus.cc:2258] T 5509924433b144f1be541f98ebb8ca97 P f211c129006447348227929f7de24c3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:13:52.912333 18707 ts_tablet_manager.cc:1905] T 5509924433b144f1be541f98ebb8ca97 P f211c129006447348227929f7de24c3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:13:52.913007 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 18266
I20260924 21:13:52.915570 18703 mvcc.cc:205] Tried to move back new op lower bound from 7333005037160378368 to 7333005037096660992. Current Snapshot: MvccSnapshot[applied={T|T < 7333005037160378368}]
I20260924 21:13:52.916157 18707 ts_tablet_manager.cc:1918] T 5509924433b144f1be541f98ebb8ca97 P f211c129006447348227929f7de24c3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:13:52.916306 18707 log.cc:1196] T 5509924433b144f1be541f98ebb8ca97 P f211c129006447348227929f7de24c3c: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284333163253-6986-0/minicluster-data/ts-2/wal/wals/5509924433b144f1be541f98ebb8ca97
I20260924 21:13:52.916632 18707 ts_tablet_manager.cc:1939] T 5509924433b144f1be541f98ebb8ca97 P f211c129006447348227929f7de24c3c: Deleting consensus metadata
I20260924 21:13:52.918720 18171 catalog_manager.cc:4841] TS f211c129006447348227929f7de24c3c (127.6.210.131:46665): tablet 5509924433b144f1be541f98ebb8ca97 (table test-table [id=6ee1811d1eb340348ad300026bde0a01]) successfully deleted
I20260924 21:13:52.918957 18709 tablet_replica.cc:330] T 5509924433b144f1be541f98ebb8ca97 P 6609fb483edd43bd8667c5cf225de678: stopping tablet replica
I20260924 21:13:52.919137 18709 raft_consensus.cc:2229] T 5509924433b144f1be541f98ebb8ca97 P 6609fb483edd43bd8667c5cf225de678 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:13:52.919261 18709 raft_consensus.cc:2258] T 5509924433b144f1be541f98ebb8ca97 P 6609fb483edd43bd8667c5cf225de678 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:13:52.925563 18709 ts_tablet_manager.cc:1905] T 5509924433b144f1be541f98ebb8ca97 P 6609fb483edd43bd8667c5cf225de678: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 21:13:52.925856 18171 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.210.129:40495 (error 108)
I20260924 21:13:52.926764 18681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:52.926619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:40495 (local address 127.0.0.1:34800)
0924 21:13:52.926682 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:52.926721 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.129:40495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260924 21:13:52.926890 18171 catalog_manager.cc:4584] TS ad4b13f27e8f4164ab698477c0742c86 (127.6.210.129:40495): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5509924433b144f1be541f98ebb8ca97: Network error: Client connection negotiation failed: client connection to 127.6.210.129:40495: connect: Connection refused (error 111)
I20260924 21:13:52.927085 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 18403
I20260924 21:13:52.933686 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 18539
I20260924 21:13:52.934504 18681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:13:52.934395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:46381 (local address 127.0.0.1:48902)
0924 21:13:52.934443 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0924 21:13:52.934467 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.130:46381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260924 21:13:52.940591 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 18152
I20260924 21:13:52.945955 6986 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260924 21:13:53.281011 6986 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260924 21:13:53.281122 6986 mini_ranger.cc:66] Stopping Ranger...
I20260924 21:13:53.625983 6986 mini_ranger.cc:68] Stopped Ranger
2026-09-24 21:13:53.626 UTC [17738] LOG: received smart shutdown request
2026-09-24 21:13:53.629 UTC [17738] LOG: background worker "logical replication launcher" (PID 17747) exited with exit code 1
2026-09-24 21:13:53.630 UTC [17742] LOG: shutting down
2026-09-24 21:13:53.798 UTC [17738] LOG: database system is shut down
2026-09-24T21:13:53Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (46056 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:13:53 dist-test-slave-q276 krb5kdc[18721](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:13:53 dist-test-slave-q276 krb5kdc[18721](info): set up 2 sockets
Sep 24 21:13:53 dist-test-slave-q276 krb5kdc[18721](info): commencing operation
krb5kdc: starting...
W20260924 21:13:55.034199 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:13:55 dist-test-slave-q276 krb5kdc[18721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:13:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:13:55Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.6.210.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.6.210.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.6.210.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/rangeradmin_127.6.210.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.6.210.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/rangeradmin_127.6.210.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.6.210.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.6.210.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.6.210.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/rangerlookup_127.6.210.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.6.210.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/rangerlookup_127.6.210.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.210.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.6.210.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.210.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/HTTP_127.6.210.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.210.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/HTTP_127.6.210.148@KRBTEST.COM.keytab.
I20260924 21:13:55.200515 6986 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task5s070Q/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/postgres -l logfile start
2026-09-24 21:13:57.770 UTC [18751] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-24 21:13:57.771 UTC [18751] LOG: listening on IPv4 address "127.6.210.148", port 47109
2026-09-24 21:13:57.774 UTC [18751] LOG: listening on Unix socket "/tmp/.s.PGSQL.47109"
2026-09-24 21:13:57.795 UTC [18754] LOG: database system was shut down at 2026-09-24 21:13:56 UTC
2026-09-24 21:13:57.799 UTC [18751] LOG: database system is ready to accept connections
I20260924 21:13:58.826853 6986 mini_postgres.cc:96] Postgres bound to 47109
2026-09-24 21:13:58.833 UTC [18762] FATAL: database "slave" does not exist
127.6.210.148:47109 - accepting connections
I20260924 21:13:58.833703 6986 mini_ranger.cc:161] Starting Ranger...
I20260924 21:13:58.850580 6986 mini_ranger.cc:84] Created miniranger Postgres user
I20260924 21:13:59.333792 6986 mini_ranger.cc:87] Created ranger Postgres database
I20260924 21:13:59.333879 6986 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ranger-admin
2026-09-24 21:13:59,612 [I] DB FLAVOR :POSTGRES
2026-09-24 21:13:59,613 [I] --------- Verifying Ranger DB connection ---------
2026-09-24 21:13:59,613 [I] Checking connection..
2026-09-24 21:13:59,613 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:47109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-24 21:13:59,855 [I] Checking connection passed.
2026-09-24 21:13:59,855 [I] --------- Verifying version history table ---------
2026-09-24 21:13:59,856 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:47109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 21:14:00,123 [I] Table x_db_version_h does not exist in database ranger
2026-09-24 21:14:00,124 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:47109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 21:14:00,376 [I] Table x_db_version_h does not exist in database ranger
2026-09-24 21:14:00,377 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-24 21:14:00,377 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:47109/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-24 21:14:00.599 UTC [18826] WARNING: there is no transaction in progress
2026-09-24 21:14:00,605 [I] create_dbversion_catalog.sql file imported successfully
2026-09-24 21:14:00,606 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:47109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 21:14:00,846 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-24 21:14:00,846 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-24 21:14:00,847 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:47109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-24 21:14:01,103 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:47109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-24 21:14:01,351 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:47109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-q276.c.gcp-upstream.internal','N') ;"
2026-09-24 21:14:01,610 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:47109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-24 21:14:01,868 [I] Table x_portal_user does not exist in database ranger
2026-09-24 21:14:01,868 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:47109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-24 21:14:02,106 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-24 21:14:02,106 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:47109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-24 21:14:02,355 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:47109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-24 21:14:02,590 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-24 21:14:02,590 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:47109/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-24 21:14:03.590 UTC [18934] WARNING: there is no transaction in progress
2026-09-24 21:14:03.609 UTC [18934] WARNING: there is no transaction in progress
2026-09-24 21:14:03.627 UTC [18934] WARNING: there is no transaction in progress
2026-09-24 21:14:03.644 UTC [18934] WARNING: there is no transaction in progress
2026-09-24 21:14:03.660 UTC [18934] WARNING: there is no transaction in progress
2026-09-24 21:14:03.681 UTC [18934] WARNING: there is no transaction in progress
2026-09-24 21:14:03.738 UTC [18934] WARNING: there is no transaction in progress
2026-09-24 21:14:03.745 UTC [18934] WARNING: there is no transaction in progress
2026-09-24 21:14:03.760 UTC [18934] WARNING: there is no transaction in progress
2026-09-24 21:14:03.771 UTC [18934] WARNING: there is no transaction in progress
2026-09-24 21:14:03.781 UTC [18934] WARNING: there is no transaction in progress
2026-09-24 21:14:03.796 UTC [18934] WARNING: there is no transaction in progress
2026-09-24 21:14:03.806 UTC [18934] WARNING: there is no transaction in progress
2026-09-24 21:14:03.812 UTC [18934] WARNING: there is no transaction in progress
2026-09-24 21:14:03.820 UTC [18934] WARNING: there is no transaction in progress
2026-09-24 21:14:03.827 UTC [18934] WARNING: there is no transaction in progress
2026-09-24 21:14:04,538 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-24 21:14:04,538 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:47109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-24 21:14:04,772 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-24 21:14:04,773 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:47109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-24 21:14:05,068 [I] Table x_portal_user already exists in database 'ranger'
2026-09-24 21:14:05,069 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:47109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-24 21:14:05,308 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-24 21:14:05,309 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:47109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-24 21:14:05,562 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:47109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-24 21:14:05,799 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:47109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-q276.c.gcp-upstream.internal';"
2026-09-24 21:14:06,024 [I] CORE_DB_SCHEMA import status has been updated
2026-09-24 21:14:06,025 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:47109/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-24 21:14:06,260 [I] DB_PATCHES have already been applied
I20260924 21:14:06.266160 6986 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ranger-admin:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task5s070Q/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260924 21:14:06.266230 6986 mini_ranger.cc:193] Using host: 127.6.210.148
I20260924 21:14:06.270044 6986 mini_ranger.cc:227] Ranger admin URL: http://127.6.210.148:45377
Sep 24, 2026 9:14:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 24, 2026 9:14:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 24, 2026 9:14:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ranger-admin/ews/webapp
Sep 24, 2026 9:14:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 24, 2026 9:14:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 24, 2026 9:14:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ranger-admin/ews/webapp].
Sep 24, 2026 9:14:07 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260924 21:14:07.384647 6986 mini_ranger.cc:160] Time spent starting Ranger: real 8.551s user 0.000s sys 0.009s
Sep 24, 2026 9:14:28 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 24, 2026 9:14:28 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 24, 2026 9:14:28 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 24, 2026 9:14:28 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 24, 2026 9:14:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 24, 2026 9:14:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 24, 2026 9:14:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 24, 2026 9:14:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 24, 2026 9:14:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 24, 2026 9:14:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 24, 2026 9:14:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 24, 2026 9:14:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 24, 2026 9:14:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 24, 2026 9:14:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 24, 2026 9:14:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 24, 2026 9:14:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 24, 2026 9:14:28 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 24, 2026 9:14:29 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260924 21:14:29.844004 6986 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.6.210.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.6.210.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.6.210.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/rangerkms_127.6.210.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.6.210.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/rangerkms_127.6.210.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.210.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.6.210.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.210.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/HTTP_127.6.210.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.210.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/HTTP_127.6.210.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 24 21:14:29 dist-test-slave-q276 krb5kdc[18721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284469, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260924 21:14:29.928975 6986 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260924 21:14:29.938896 6986 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260924 21:14:30.909174 6986 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260924 21:14:30.909263 6986 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ranger-kms
I20260924 21:14:30.909284 6986 mini_ranger_kms.cc:227] Using postgres at 127.6.210.148:47109
2026-09-24 21:14:31,076 [I] DB FLAVOR :POSTGRES
2026-09-24 21:14:31,076 [I] --------- Verifying Ranger DB connection ---------
2026-09-24 21:14:31,076 [I] Checking connection
2026-09-24 21:14:31,076 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:47109/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-24 21:14:31,357 [I] connection success
2026-09-24 21:14:31,357 [I] --------- Verifying Ranger DB tables ---------
2026-09-24 21:14:31,358 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-24 21:14:31,358 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:47109/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-24 21:14:31,663 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-24 21:14:31,663 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-24 21:14:31,663 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-24 21:14:31,663 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.210.148:47109/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-24 21:14:31,998 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260924 21:14:32.475625 6986 mini_ranger_kms.cc:310] Created kms service
I20260924 21:14:32.657387 6986 mini_ranger_kms.cc:326] Created kudu user
I20260924 21:14:32.700577 6986 mini_ranger_kms.cc:343] Created rangerkms user
Sep 24 21:14:32 dist-test-slave-q276 krb5kdc[18721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284469, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.6.210.148@KRBTEST.COM
I20260924 21:14:33.120090 6986 mini_ranger_kms.cc:384] Added ranger policy
I20260924 21:14:33.120301 6986 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ranger-kms:/tmp/dist-test-task5s070Q/build/release/bin/postgresql.jar:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task5s070Q/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260924 21:14:33.120322 6986 mini_ranger_kms.cc:240] Using host: 127.6.210.148
I20260924 21:14:33.124275 6986 mini_ranger_kms.cc:276] Ranger KMS PID: 19115
I20260924 21:14:33.124353 6986 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.6.210.148:57969
Sep 24, 2026 9:14:34 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 24, 2026 9:14:34 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 24, 2026 9:14:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 24, 2026 9:14:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 24, 2026 9:14:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ranger-kms/ews/webapp].
Sep 24, 2026 9:14:34 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Sep 24 21:14:35 dist-test-slave-q276 krb5kdc[18721](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790284475, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.210.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:14:35.474550 6986 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.546s user 0.008s sys 0.140s
I20260924 21:14:35.474653 6986 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260924 21:14:35.474702 6986 mini_ranger_kms.cc:401] 127.6.210.148:57969/kms/v1/keys
Sep 24 21:14:35 dist-test-slave-q276 krb5kdc[18721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284475, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.210.148@KRBTEST.COM for HTTP/127.6.210.148@KRBTEST.COM
Sep 24 21:14:36 dist-test-slave-q276 krb5kdc[18721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284475, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.210.148@KRBTEST.COM for HTTP/127.6.210.148@KRBTEST.COM
Sep 24 21:14:38 dist-test-slave-q276 krb5kdc[18721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.6.210.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
I20260924 21:14:38.582077 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:42429
--webserver_interface=127.6.210.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:33871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:42429
--ranger_config_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.210.148:57969/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:14:38.685618 19165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:14:38.685859 19165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:14:38.685882 19165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:14:38.687077 19165 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:14:38.687124 19165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:14:38.687139 19165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:14:38.687152 19165 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:14:38.687165 19165 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:14:38.688746 19165 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:33871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/master-0/wal
--ranger_kms_url=127.6.210.148:57969/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:42429
--ranger_config_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:42429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:14:38.688939 19165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:14:38.689108 19165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:14:38.694380 19170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:14:38.694403 19165 server_base.cc:1034] running on GCE node
W20260924 21:14:38.694386 19173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:38.694401 19171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:14:38.694868 19165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:14:38.695175 19165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:14:38.696341 19165 hybrid_clock.cc:648] HybridClock initialized: now 1790284478696315 us; error 60 us; skew 500 ppm
Sep 24 21:14:38 dist-test-slave-q276 krb5kdc[18721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284478, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:14:38.703094 19165 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:14:38.703527 19165 webserver.cc:460] Webserver started at http://127.6.210.190:37965/ using document root <none> and password file <none>
I20260924 21:14:38.703713 19165 fs_manager.cc:359] Metadata directory not provided
I20260924 21:14:38.703750 19165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:14:38.703843 19165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 21:14:38 dist-test-slave-q276 krb5kdc[18721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284478, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for HTTP/127.6.210.148@KRBTEST.COM
I20260924 21:14:38.754554 19165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "e8e01028296147919455f083c3e2d7e9"
format_stamp: "Formatted at 2026-09-24 21:14:38 on dist-test-slave-q276"
server_key: "af6fa3f2614001ff312c4b78541b03e7"
server_key_iv: "bec6e99715b1099e3cd3d97eef594537"
server_key_version: "kuduclusterkey@0"
I20260924 21:14:38.755049 19165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "e8e01028296147919455f083c3e2d7e9"
format_stamp: "Formatted at 2026-09-24 21:14:38 on dist-test-slave-q276"
server_key: "af6fa3f2614001ff312c4b78541b03e7"
server_key_iv: "bec6e99715b1099e3cd3d97eef594537"
server_key_version: "kuduclusterkey@0"
I20260924 21:14:38.756970 19165 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260924 21:14:38.772965 19180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:38.773231 19165 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:14:38.773353 19165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "e8e01028296147919455f083c3e2d7e9"
format_stamp: "Formatted at 2026-09-24 21:14:38 on dist-test-slave-q276"
server_key: "af6fa3f2614001ff312c4b78541b03e7"
server_key_iv: "bec6e99715b1099e3cd3d97eef594537"
server_key_version: "kuduclusterkey@0"
I20260924 21:14:38.773469 19165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:14:38.818018 19165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:14:38.820009 19165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:14:38.820161 19165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:14:38.824787 19165 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:42429
I20260924 21:14:38.824826 19232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:42429 every 8 connection(s)
I20260924 21:14:38.825191 19165 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:14:38.825978 19233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:14:38.829023 19233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9: Bootstrap starting.
I20260924 21:14:38.829336 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 19165
I20260924 21:14:38.829448 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:14:38.829624 6986 external_mini_cluster.cc:1428] Setting key 854589d84b6a2bd51b0661527e3129cd
I20260924 21:14:38.829747 19233 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9: Neither blocks nor log segments found. Creating new log.
I20260924 21:14:38.830317 19233 log.cc:824] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9: Log is configured to *not* fsync() on all Append() calls
I20260924 21:14:38.831171 19233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9: No bootstrap required, opened a new log
I20260924 21:14:38.833703 19233 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e01028296147919455f083c3e2d7e9" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 42429 } }
I20260924 21:14:38.833889 19233 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:14:38.833925 19233 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8e01028296147919455f083c3e2d7e9, State: Initialized, Role: FOLLOWER
I20260924 21:14:38.834095 19233 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e01028296147919455f083c3e2d7e9" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 42429 } }
I20260924 21:14:38.834290 19233 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:14:38.834347 19233 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:14:38.834391 19233 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:14:38.835100 19233 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e01028296147919455f083c3e2d7e9" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 42429 } }
I20260924 21:14:38.835214 19233 leader_election.cc:302] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8e01028296147919455f083c3e2d7e9; no voters:
I20260924 21:14:38.835418 19233 leader_election.cc:288] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:14:38.835480 19238 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:14:38.835670 19238 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [term 1 LEADER]: Becoming Leader. State: Replica: e8e01028296147919455f083c3e2d7e9, State: Running, Role: LEADER
I20260924 21:14:38.835764 19233 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:14:38.835873 19238 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e01028296147919455f083c3e2d7e9" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 42429 } }
I20260924 21:14:38.836292 19238 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8e01028296147919455f083c3e2d7e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e01028296147919455f083c3e2d7e9" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 42429 } } }
I20260924 21:14:38.836431 19238 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [sys.catalog]: This master's current role is: LEADER
Sep 24 21:14:38 dist-test-slave-q276 krb5kdc[18721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:14:38.836619 19238 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8e01028296147919455f083c3e2d7e9. Latest consensus state: current_term: 1 leader_uuid: "e8e01028296147919455f083c3e2d7e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e01028296147919455f083c3e2d7e9" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 42429 } } }
I20260924 21:14:38.836738 19238 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [sys.catalog]: This master's current role is: LEADER
I20260924 21:14:38.836927 19247 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:14:38.837200 19233 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260924 21:14:38.841224 19247 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:14:38.843267 19236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:38.830152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50774 (local address 127.6.210.190:42429)
0924 21:14:38.830352 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:38.830363 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:38.830397 (+ 34us) server_negotiation.cc:407] Connection header received
0924 21:14:38.831022 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:38.831031 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:38.831086 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:38.831243 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:38.831912 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:38.832797 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:38.833560 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:38.834076 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:38.834182 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:38.837026 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:38.837031 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:38.837031 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:38.837044 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:38.841490 (+ 4446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:38.841938 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:38.841941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:38.841941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:38.842051 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:38.842250 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:38.842251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:38.842251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:38.842604 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:38.842775 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:38.842870 (+ 95us) server_negotiation.cc:299] Negotiation successful
0924 21:14:38.843041 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":82,"threads_started":1}
I20260924 21:14:38.848486 19247 catalog_manager.cc:1347] Generated new cluster ID: 30a048f2889d43739187f9bdaab81fb2
I20260924 21:14:38.848613 19247 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:14:38.866811 19247 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:14:38.867681 19247 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:14:38.877869 19247 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9: Generated new TSK 0
I20260924 21:14:38.878180 19247 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.210.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
I20260924 21:14:39.608959 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.210.148:57969/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.129:0
--local_ip_for_outbound_sockets=127.6.210.129
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:42429
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:33871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:14:39.720036 19278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:14:39.720309 19278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:14:39.720343 19278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:14:39.722091 19278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:14:39.722191 19278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:14:39.722285 19278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.129
I20260924 21:14:39.724378 19278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:33871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.6.210.148:57969/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:42429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.129
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:14:39.724771 19278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:14:39.725061 19278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:14:39.730748 19286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:14:39.730917 19278 server_base.cc:1034] running on GCE node
W20260924 21:14:39.731096 19284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:39.731225 19283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:14:39.731459 19278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:14:39.731758 19278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:14:39.732945 19278 hybrid_clock.cc:648] HybridClock initialized: now 1790284479732909 us; error 50 us; skew 500 ppm
Sep 24 21:14:39 dist-test-slave-q276 krb5kdc[18721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284479, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:14:39.736331 19278 init.cc:377] Logged in from keytab as kudu/127.6.210.129@KRBTEST.COM (short username kudu)
I20260924 21:14:39.736979 19278 webserver.cc:460] Webserver started at http://127.6.210.129:32931/ using document root <none> and password file <none>
I20260924 21:14:39.737289 19278 fs_manager.cc:359] Metadata directory not provided
I20260924 21:14:39.737382 19278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:14:39.737537 19278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 21:14:39 dist-test-slave-q276 krb5kdc[18721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284479, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for HTTP/127.6.210.148@KRBTEST.COM
I20260924 21:14:39.751128 19278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-0/data/instance:
uuid: "b3349cb471c04bb1835c530c725612b3"
format_stamp: "Formatted at 2026-09-24 21:14:39 on dist-test-slave-q276"
server_key: "df4dbd34e82554a3db323f3552c64b52"
server_key_iv: "8466176c7c484194d5f81dfb620ae808"
server_key_version: "kuduclusterkey@0"
I20260924 21:14:39.751595 19278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance:
uuid: "b3349cb471c04bb1835c530c725612b3"
format_stamp: "Formatted at 2026-09-24 21:14:39 on dist-test-slave-q276"
server_key: "df4dbd34e82554a3db323f3552c64b52"
server_key_iv: "8466176c7c484194d5f81dfb620ae808"
server_key_version: "kuduclusterkey@0"
I20260924 21:14:39.753360 19278 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260924 21:14:39.769387 19295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:39.769637 19278 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:14:39.769753 19278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-0/wal
uuid: "b3349cb471c04bb1835c530c725612b3"
format_stamp: "Formatted at 2026-09-24 21:14:39 on dist-test-slave-q276"
server_key: "df4dbd34e82554a3db323f3552c64b52"
server_key_iv: "8466176c7c484194d5f81dfb620ae808"
server_key_version: "kuduclusterkey@0"
I20260924 21:14:39.769857 19278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:14:39.784536 19278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:14:39.786859 19278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:14:39.787041 19278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:14:39.787325 19278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:14:39.787716 19278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:14:39.787770 19278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:39.787838 19278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:14:39.787882 19278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:39.795522 19278 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.129:35707
I20260924 21:14:39.795742 19408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.129:35707 every 8 connection(s)
I20260924 21:14:39.796054 19278 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
Sep 24 21:14:39 dist-test-slave-q276 krb5kdc[18721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284479, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:14:39.805308 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 19278
I20260924 21:14:39.805423 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance
I20260924 21:14:39.805637 6986 external_mini_cluster.cc:1428] Setting key f567971ec20f7e89f118151f78ec6178
I20260924 21:14:39.809209 19412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:39.796898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:42429 (local address 127.6.210.129:57605)
0924 21:14:39.797090 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:39.797139 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:39.797673 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:39.798078 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:39.798083 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:39.798415 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:14:39.798724 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:39.798733 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:39.799793 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:39.799795 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:39.801222 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:39.801225 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:39.801801 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:39.801802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:39.801964 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:39.802645 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:14:39.802657 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:14:39.804412 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:14:39.807361 (+ 2949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:39.807369 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:39.807372 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:39.807705 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:39.807964 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:39.807965 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:39.807965 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:39.808117 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:39.808452 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:14:39.808459 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:14:39.808639 (+ 180us) client_negotiation.cc:769] Sending connection context
0924 21:14:39.808759 (+ 120us) client_negotiation.cc:240] Negotiation successful
0924 21:14:39.808937 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":106,"threads_started":1}
I20260924 21:14:39.809703 19411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:39.796927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:57605 (local address 127.6.210.190:42429)
0924 21:14:39.797831 (+ 904us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:39.797833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:39.797840 (+ 7us) server_negotiation.cc:407] Connection header received
0924 21:14:39.797862 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:39.797862 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:39.797874 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:39.797928 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:39.798858 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:39.799673 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:39.801343 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:39.801677 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:39.801756 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:39.804534 (+ 2778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:39.804536 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:39.804537 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:39.804546 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:39.807234 (+ 2688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:39.807815 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:39.807816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:39.807817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:39.807888 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:39.808199 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:39.808201 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:39.808201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:39.808358 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:39.808423 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:39.809504 (+ 1081us) server_negotiation.cc:299] Negotiation successful
0924 21:14:39.809598 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":875,"thread_start_us":103,"threads_started":1}
I20260924 21:14:39.810235 19409 heartbeater.cc:347] Connected to a master server at 127.6.210.190:42429
I20260924 21:14:39.810398 19409 heartbeater.cc:464] Registering TS with master...
I20260924 21:14:39.810597 19409 heartbeater.cc:511] Master 127.6.210.190:42429 requested a full tablet report, sending...
I20260924 21:14:39.811196 19197 ts_manager.cc:194] Registered new tserver with Master: b3349cb471c04bb1835c530c725612b3 (127.6.210.129:35707)
I20260924 21:14:39.812181 19197 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.129@KRBTEST.COM'} at 127.6.210.129:57605
WARNING: no policy specified for kudu/127.6.210.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
I20260924 21:14:39.835474 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.210.148:57969/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.130:0
--local_ip_for_outbound_sockets=127.6.210.130
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:42429
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:33871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:14:39.951380 19415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:14:39.951707 19415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:14:39.951772 19415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:14:39.953477 19415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:14:39.953538 19415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:14:39.953580 19415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.130
I20260924 21:14:39.955708 19415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:33871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.6.210.148:57969/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:42429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.130
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:14:39.955963 19415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:14:39.956264 19415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Sep 24 21:14:39 dist-test-slave-q276 krb5kdc[18721](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790284479, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260924 21:14:39.963069 19422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:39.963351 19425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:39.963549 19423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:14:39.964368 19415 server_base.cc:1034] running on GCE node
I20260924 21:14:39.964605 19415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:14:39.964901 19415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:14:39.966229 19415 hybrid_clock.cc:648] HybridClock initialized: now 1790284479966182 us; error 34 us; skew 500 ppm
Sep 24 21:14:39 dist-test-slave-q276 krb5kdc[18721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284479, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:14:39.969604 19415 init.cc:377] Logged in from keytab as kudu/127.6.210.130@KRBTEST.COM (short username kudu)
I20260924 21:14:39.970153 19415 webserver.cc:460] Webserver started at http://127.6.210.130:46661/ using document root <none> and password file <none>
I20260924 21:14:39.970472 19415 fs_manager.cc:359] Metadata directory not provided
I20260924 21:14:39.970597 19415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:14:39.970813 19415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 21:14:39 dist-test-slave-q276 krb5kdc[18721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284479, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for HTTP/127.6.210.148@KRBTEST.COM
I20260924 21:14:39.991981 19415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-1/data/instance:
uuid: "05d698e68b6c4fcf834ab6dc64702f85"
format_stamp: "Formatted at 2026-09-24 21:14:39 on dist-test-slave-q276"
server_key: "7b1610be74e4b1fef643a119bd1caa6f"
server_key_iv: "ef05818642754d4de70457d01b1d7021"
server_key_version: "kuduclusterkey@0"
I20260924 21:14:39.992551 19415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance:
uuid: "05d698e68b6c4fcf834ab6dc64702f85"
format_stamp: "Formatted at 2026-09-24 21:14:39 on dist-test-slave-q276"
server_key: "7b1610be74e4b1fef643a119bd1caa6f"
server_key_iv: "ef05818642754d4de70457d01b1d7021"
server_key_version: "kuduclusterkey@0"
I20260924 21:14:39.996135 19415 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260924 21:14:40.012964 19432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:40.013176 19415 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260924 21:14:40.013307 19415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-1/wal
uuid: "05d698e68b6c4fcf834ab6dc64702f85"
format_stamp: "Formatted at 2026-09-24 21:14:39 on dist-test-slave-q276"
server_key: "7b1610be74e4b1fef643a119bd1caa6f"
server_key_iv: "ef05818642754d4de70457d01b1d7021"
server_key_version: "kuduclusterkey@0"
I20260924 21:14:40.013432 19415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:14:40.027556 19415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:14:40.031281 19415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:14:40.031570 19415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:14:40.031886 19415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:14:40.032346 19415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:14:40.032409 19415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:40.032482 19415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:14:40.032516 19415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:40.040273 19415 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.130:43753
I20260924 21:14:40.040613 19546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.130:43753 every 8 connection(s)
I20260924 21:14:40.040812 19415 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
I20260924 21:14:40.046753 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 19415
I20260924 21:14:40.046864 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance
I20260924 21:14:40.047058 6986 external_mini_cluster.cc:1428] Setting key 513c3a945ece9bd4dc698b3397368045
Sep 24 21:14:40 dist-test-slave-q276 krb5kdc[18721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284479, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:14:40.053944 19411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:40.041602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:49631 (local address 127.6.210.190:42429)
0924 21:14:40.041875 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:40.041878 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:40.041932 (+ 54us) server_negotiation.cc:407] Connection header received
0924 21:14:40.042667 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:40.042669 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:40.042685 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:40.042720 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:40.043671 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:40.044569 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:40.046314 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:40.046638 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:40.046692 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:40.049982 (+ 3290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:40.049984 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:40.049984 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:40.049993 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:40.052049 (+ 2056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:40.052630 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:40.052631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:40.052631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:40.052700 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:40.053043 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:40.053044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:40.053044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:40.053202 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:40.053249 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:40.053695 (+ 446us) server_negotiation.cc:299] Negotiation successful
0924 21:14:40.053818 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244}
I20260924 21:14:40.054769 19549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:40.041611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:42429 (local address 127.6.210.130:49631)
0924 21:14:40.041802 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:40.041849 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:40.042524 (+ 675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:40.042949 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:40.042954 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:40.043222 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:14:40.043538 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:40.043545 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:40.044709 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:40.044710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:40.046159 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:40.046162 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:40.046813 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:40.046814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:40.046977 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:40.047944 (+ 967us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:14:40.047955 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:14:40.049845 (+ 1890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:14:40.052165 (+ 2320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:40.052173 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:40.052176 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:40.052523 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:40.052796 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:40.052796 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:40.052797 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:40.052956 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:40.053325 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:14:40.053332 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:14:40.053524 (+ 192us) client_negotiation.cc:769] Sending connection context
0924 21:14:40.054235 (+ 711us) client_negotiation.cc:240] Negotiation successful
0924 21:14:40.054483 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":104,"threads_started":1}
I20260924 21:14:40.055382 19547 heartbeater.cc:347] Connected to a master server at 127.6.210.190:42429
I20260924 21:14:40.055485 19547 heartbeater.cc:464] Registering TS with master...
I20260924 21:14:40.055689 19547 heartbeater.cc:511] Master 127.6.210.190:42429 requested a full tablet report, sending...
I20260924 21:14:40.056128 19197 ts_manager.cc:194] Registered new tserver with Master: 05d698e68b6c4fcf834ab6dc64702f85 (127.6.210.130:43753)
I20260924 21:14:40.056672 19197 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.130@KRBTEST.COM'} at 127.6.210.130:49631
WARNING: no policy specified for kudu/127.6.210.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
I20260924 21:14:40.093794 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.210.148:57969/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.131:0
--local_ip_for_outbound_sockets=127.6.210.131
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:42429
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:33871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:14:40.223278 19552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:14:40.223636 19552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:14:40.223735 19552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:14:40.225529 19552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:14:40.225745 19552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:14:40.225842 19552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.131
I20260924 21:14:40.228232 19552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:33871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.6.210.148:57969/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:42429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.131
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:14:40.228713 19552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:14:40.229033 19552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:14:40.235741 19560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:40.236449 19557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:40.237016 19558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:14:40.237298 19552 server_base.cc:1034] running on GCE node
I20260924 21:14:40.237715 19552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:14:40.237990 19552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:14:40.239158 19552 hybrid_clock.cc:648] HybridClock initialized: now 1790284480239122 us; error 54 us; skew 500 ppm
Sep 24 21:14:40 dist-test-slave-q276 krb5kdc[18721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284480, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:14:40.242337 19552 init.cc:377] Logged in from keytab as kudu/127.6.210.131@KRBTEST.COM (short username kudu)
I20260924 21:14:40.242820 19552 webserver.cc:460] Webserver started at http://127.6.210.131:35325/ using document root <none> and password file <none>
I20260924 21:14:40.243052 19552 fs_manager.cc:359] Metadata directory not provided
I20260924 21:14:40.243098 19552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:14:40.243214 19552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 21:14:40 dist-test-slave-q276 krb5kdc[18721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284480, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for HTTP/127.6.210.148@KRBTEST.COM
I20260924 21:14:40.261574 19552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-2/data/instance:
uuid: "9405156f66fc40b99437f80bcb102367"
format_stamp: "Formatted at 2026-09-24 21:14:40 on dist-test-slave-q276"
server_key: "e05d2f133ad02fc2bc92b2609da73c12"
server_key_iv: "f4090d199ebf3570e0e95ad935b166ae"
server_key_version: "kuduclusterkey@0"
I20260924 21:14:40.262131 19552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance:
uuid: "9405156f66fc40b99437f80bcb102367"
format_stamp: "Formatted at 2026-09-24 21:14:40 on dist-test-slave-q276"
server_key: "e05d2f133ad02fc2bc92b2609da73c12"
server_key_iv: "f4090d199ebf3570e0e95ad935b166ae"
server_key_version: "kuduclusterkey@0"
I20260924 21:14:40.270690 19552 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.000s
I20260924 21:14:40.284276 19567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:40.284729 19552 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:14:40.284842 19552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-2/wal
uuid: "9405156f66fc40b99437f80bcb102367"
format_stamp: "Formatted at 2026-09-24 21:14:40 on dist-test-slave-q276"
server_key: "e05d2f133ad02fc2bc92b2609da73c12"
server_key_iv: "f4090d199ebf3570e0e95ad935b166ae"
server_key_version: "kuduclusterkey@0"
I20260924 21:14:40.284935 19552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:14:40.299413 19552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:14:40.301837 19552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:14:40.301983 19552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:14:40.302227 19552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:14:40.302626 19552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:14:40.302672 19552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:40.302702 19552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:14:40.302775 19552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:40.317978 19552 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.131:39001
I20260924 21:14:40.318336 19680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.131:39001 every 8 connection(s)
I20260924 21:14:40.318535 19552 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
I20260924 21:14:40.325845 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 19552
I20260924 21:14:40.325965 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance
I20260924 21:14:40.326161 6986 external_mini_cluster.cc:1428] Setting key ca77053910fa05e896b8984ab78d1638
Sep 24 21:14:40 dist-test-slave-q276 krb5kdc[18721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284480, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:14:40.333212 19411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:40.319574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:45495 (local address 127.6.210.190:42429)
0924 21:14:40.319623 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:40.319626 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:40.322632 (+ 3006us) server_negotiation.cc:407] Connection header received
0924 21:14:40.323355 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:40.323357 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:40.323371 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:40.323404 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:40.324243 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:40.324960 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:40.326653 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:40.326957 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:40.327003 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:40.329968 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:40.329969 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:40.329970 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:40.329977 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:40.331656 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:40.332148 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:40.332149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:40.332150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:40.332206 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:40.332497 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:40.332497 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:40.332498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:40.332630 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:40.332670 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:40.333045 (+ 375us) server_negotiation.cc:299] Negotiation successful
0924 21:14:40.333125 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 21:14:40.333963 19683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:40.319268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:42429 (local address 127.6.210.131:45495)
0924 21:14:40.322458 (+ 3190us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:40.322519 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:40.323211 (+ 692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:40.323597 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:40.323601 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:40.323859 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:14:40.324134 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:40.324140 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:40.325086 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:40.325087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:40.326560 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:40.326563 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:40.327279 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:40.327280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:40.327438 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:40.328166 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:14:40.328178 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:14:40.329856 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:14:40.331746 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:40.331753 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:40.331756 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:40.332059 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:40.332272 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:40.332273 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:40.332273 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:40.332422 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:40.332716 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:14:40.332721 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:14:40.332889 (+ 168us) client_negotiation.cc:769] Sending connection context
0924 21:14:40.333559 (+ 670us) client_negotiation.cc:240] Negotiation successful
0924 21:14:40.333730 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3133,"thread_start_us":104,"threads_started":1}
I20260924 21:14:40.334744 19681 heartbeater.cc:347] Connected to a master server at 127.6.210.190:42429
I20260924 21:14:40.334887 19681 heartbeater.cc:464] Registering TS with master...
I20260924 21:14:40.335114 19681 heartbeater.cc:511] Master 127.6.210.190:42429 requested a full tablet report, sending...
I20260924 21:14:40.335526 19197 ts_manager.cc:194] Registered new tserver with Master: 9405156f66fc40b99437f80bcb102367 (127.6.210.131:39001)
I20260924 21:14:40.336009 19197 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.131@KRBTEST.COM'} at 127.6.210.131:45495
I20260924 21:14:40.338184 6986 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:14:40.338863 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 19278
I20260924 21:14:40.343848 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 19415
I20260924 21:14:40.349318 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 19552
I20260924 21:14:40.354668 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 19165
I20260924 21:14:40.359436 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:42429
--webserver_interface=127.6.210.190
--webserver_port=37965
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:33871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:42429
--ranger_config_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.210.148:57969/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.210.148:57969/kms with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:14:40.443186 19684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:14:40.443435 19684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:14:40.443457 19684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:14:40.444690 19684 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:14:40.444738 19684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:14:40.444753 19684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:14:40.444765 19684 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:14:40.444777 19684 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:14:40.446532 19684 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:33871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.210.148:57969/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:42429
--ranger_config_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:42429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=37965
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:14:40.446786 19684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:14:40.447034 19684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:14:40.452816 19689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:40.452849 19690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:40.452817 19692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:14:40.453296 19684 server_base.cc:1034] running on GCE node
I20260924 21:14:40.453560 19684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:14:40.453912 19684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:14:40.455076 19684 hybrid_clock.cc:648] HybridClock initialized: now 1790284480455050 us; error 38 us; skew 500 ppm
Sep 24 21:14:40 dist-test-slave-q276 krb5kdc[18721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284480, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:14:40.458297 19684 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:14:40.458832 19684 webserver.cc:460] Webserver started at http://127.6.210.190:37965/ using document root <none> and password file <none>
I20260924 21:14:40.459106 19684 fs_manager.cc:359] Metadata directory not provided
I20260924 21:14:40.459167 19684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 24 21:14:40 dist-test-slave-q276 krb5kdc[18721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284480, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for HTTP/127.6.210.148@KRBTEST.COM
I20260924 21:14:40.517795 19684 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:14:40.518573 19700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:40.518761 19684 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:14:40.518848 19684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "e8e01028296147919455f083c3e2d7e9"
format_stamp: "Formatted at 2026-09-24 21:14:38 on dist-test-slave-q276"
server_key: "af6fa3f2614001ff312c4b78541b03e7"
server_key_iv: "bec6e99715b1099e3cd3d97eef594537"
server_key_version: "kuduclusterkey@0"
I20260924 21:14:40.519287 19684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:14:40.533249 19684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:14:40.535485 19684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:14:40.535661 19684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:14:40.540702 19684 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:42429
I20260924 21:14:40.540791 19752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:42429 every 8 connection(s)
I20260924 21:14:40.541188 19684 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:14:40.542588 19753 sys_catalog.cc:263] Verifying existing consensus state
I20260924 21:14:40.543268 19753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9: Bootstrap starting.
I20260924 21:14:40.546133 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 19684
I20260924 21:14:40.546487 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.210.148:57969/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.129:35707
--local_ip_for_outbound_sockets=127.6.210.129
--tserver_master_addrs=127.6.210.190:42429
--webserver_port=32931
--webserver_interface=127.6.210.129
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:33871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.210.148:57969/kms with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:14:40.552713 19753 log.cc:824] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9: Log is configured to *not* fsync() on all Append() calls
I20260924 21:14:40.555307 19753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 21:14:40.555656 19753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9: Bootstrap complete.
I20260924 21:14:40.559567 19753 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e01028296147919455f083c3e2d7e9" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 42429 } }
I20260924 21:14:40.559893 19753 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8e01028296147919455f083c3e2d7e9, State: Initialized, Role: FOLLOWER
I20260924 21:14:40.560058 19753 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e01028296147919455f083c3e2d7e9" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 42429 } }
I20260924 21:14:40.560166 19753 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:14:40.560237 19753 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:14:40.560288 19753 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [term 1 FOLLOWER]: Advancing to term 2
I20260924 21:14:40.561638 19753 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e01028296147919455f083c3e2d7e9" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 42429 } }
I20260924 21:14:40.561749 19753 leader_election.cc:302] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8e01028296147919455f083c3e2d7e9; no voters:
I20260924 21:14:40.561940 19753 leader_election.cc:288] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 21:14:40.562007 19757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 21:14:40.562213 19757 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [term 2 LEADER]: Becoming Leader. State: Replica: e8e01028296147919455f083c3e2d7e9, State: Running, Role: LEADER
I20260924 21:14:40.562299 19753 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:14:40.562381 19757 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e01028296147919455f083c3e2d7e9" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 42429 } }
I20260924 21:14:40.562734 19757 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e8e01028296147919455f083c3e2d7e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e01028296147919455f083c3e2d7e9" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 42429 } } }
I20260924 21:14:40.562858 19757 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [sys.catalog]: This master's current role is: LEADER
I20260924 21:14:40.563012 19757 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8e01028296147919455f083c3e2d7e9. Latest consensus state: current_term: 2 leader_uuid: "e8e01028296147919455f083c3e2d7e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8e01028296147919455f083c3e2d7e9" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 42429 } } }
I20260924 21:14:40.563102 19757 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9 [sys.catalog]: This master's current role is: LEADER
I20260924 21:14:40.563228 19766 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:14:40.564164 19753 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260924 21:14:40.564427 19766 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:14:40.568434 19766 catalog_manager.cc:1259] Loaded cluster ID: 30a048f2889d43739187f9bdaab81fb2
I20260924 21:14:40.568599 19766 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:14:40.570406 19766 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:14:40.570672 19766 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9: Loaded TSK: 0
I20260924 21:14:40.570951 19766 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 21:14:40.665911 19755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:14:40.666190 19755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:14:40.666229 19755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:14:40.668005 19755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:14:40.668072 19755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:14:40.668112 19755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.129
I20260924 21:14:40.670315 19755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:33871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.210.148:57969/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.129:35707
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.210.129
--webserver_port=32931
--tserver_master_addrs=127.6.210.190:42429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.129
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:14:40.670575 19755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:14:40.670893 19755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:14:40.677304 19793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:14:40.677958 19755 server_base.cc:1034] running on GCE node
W20260924 21:14:40.677394 19791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:40.677557 19790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:14:40.678440 19755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:14:40.678748 19755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:14:40.679941 19755 hybrid_clock.cc:648] HybridClock initialized: now 1790284480679904 us; error 50 us; skew 500 ppm
Sep 24 21:14:40 dist-test-slave-q276 krb5kdc[18721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284480, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:14:40.683260 19755 init.cc:377] Logged in from keytab as kudu/127.6.210.129@KRBTEST.COM (short username kudu)
I20260924 21:14:40.683811 19755 webserver.cc:460] Webserver started at http://127.6.210.129:32931/ using document root <none> and password file <none>
I20260924 21:14:40.684072 19755 fs_manager.cc:359] Metadata directory not provided
I20260924 21:14:40.684144 19755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 24 21:14:40 dist-test-slave-q276 krb5kdc[18721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284480, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for HTTP/127.6.210.148@KRBTEST.COM
I20260924 21:14:40.712960 19755 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:14:40.718024 19801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:40.718314 19755 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.000s
I20260924 21:14:40.718413 19755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-0/wal
uuid: "b3349cb471c04bb1835c530c725612b3"
format_stamp: "Formatted at 2026-09-24 21:14:39 on dist-test-slave-q276"
server_key: "df4dbd34e82554a3db323f3552c64b52"
server_key_iv: "8466176c7c484194d5f81dfb620ae808"
server_key_version: "kuduclusterkey@0"
I20260924 21:14:40.718905 19755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:14:40.732055 19755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:14:40.734509 19755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:14:40.734720 19755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:14:40.735015 19755 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:14:40.735422 19755 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:14:40.735534 19755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:40.735636 19755 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:14:40.735713 19755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:40.741683 19755 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.129:35707
I20260924 21:14:40.742182 19755 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
I20260924 21:14:40.745016 19914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.129:35707 every 8 connection(s)
I20260924 21:14:40.745191 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 19755
I20260924 21:14:40.745487 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.210.148:57969/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.130:43753
--local_ip_for_outbound_sockets=127.6.210.130
--tserver_master_addrs=127.6.210.190:42429
--webserver_port=46661
--webserver_interface=127.6.210.130
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:33871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.210.148:57969/kms with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 21:14:40 dist-test-slave-q276 krb5kdc[18721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284480, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:14:40.772732 19919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:40.748514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:46281 (local address 127.6.210.190:42429)
0924 21:14:40.749722 (+ 1208us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:40.749732 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:40.749769 (+ 37us) server_negotiation.cc:407] Connection header received
0924 21:14:40.749911 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:40.749916 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:40.749971 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:40.750102 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:40.750905 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:40.752382 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:40.755107 (+ 2725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:40.755603 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:40.762356 (+ 6753us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:40.762457 (+ 101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:40.762461 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:40.762461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:40.762471 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:40.764893 (+ 2422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:40.770958 (+ 6065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:40.770963 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:40.770963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:40.771098 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:40.771433 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:40.771434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:40.771435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:40.771773 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:40.771869 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:40.772304 (+ 435us) server_negotiation.cc:299] Negotiation successful
0924 21:14:40.772455 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1150,"thread_start_us":81,"threads_started":1}
I20260924 21:14:40.773741 19918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:40.748420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:42429 (local address 127.6.210.129:46281)
0924 21:14:40.748724 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:40.748775 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:40.749507 (+ 732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:40.750207 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:40.750212 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:40.750479 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:14:40.750753 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:40.750760 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:40.752529 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:40.752530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:40.754985 (+ 2455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:40.754988 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:40.755734 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:40.755736 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:40.755875 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:40.756491 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:14:40.756503 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:14:40.758493 (+ 1990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:14:40.770353 (+ 11860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:40.770368 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:40.770370 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:40.770766 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:40.771186 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:40.771187 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:40.771188 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:40.771342 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:40.771912 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:14:40.771919 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:14:40.772093 (+ 174us) client_negotiation.cc:769] Sending connection context
0924 21:14:40.773276 (+ 1183us) client_negotiation.cc:240] Negotiation successful
0924 21:14:40.773480 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":92,"threads_started":1}
I20260924 21:14:40.774741 19915 heartbeater.cc:347] Connected to a master server at 127.6.210.190:42429
I20260924 21:14:40.774852 19915 heartbeater.cc:464] Registering TS with master...
I20260924 21:14:40.775048 19915 heartbeater.cc:511] Master 127.6.210.190:42429 requested a full tablet report, sending...
W20260924 21:14:40.776047 19915 heartbeater.cc:644] Failed to heartbeat to 127.6.210.190:42429 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260924 21:14:40.776122 19915 heartbeater.cc:464] Registering TS with master...
I20260924 21:14:40.776188 19915 heartbeater.cc:511] Master 127.6.210.190:42429 requested a full tablet report, sending...
I20260924 21:14:40.776506 19915 heartbeater.cc:464] Registering TS with master...
I20260924 21:14:40.776572 19915 heartbeater.cc:511] Master 127.6.210.190:42429 requested a full tablet report, sending...
W20260924 21:14:40.776881 19915 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 21:14:40.863686 19917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:14:40.864041 19917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:14:40.864130 19917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:14:40.865955 19917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:14:40.866067 19917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:14:40.866130 19917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.130
I20260924 21:14:40.868366 19917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:33871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.210.148:57969/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.130:43753
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.210.130
--webserver_port=46661
--tserver_master_addrs=127.6.210.190:42429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.130
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:14:40.868619 19917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:14:40.868923 19917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:14:40.875274 19924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:40.875373 19925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:40.875259 19927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:14:40.875598 19917 server_base.cc:1034] running on GCE node
I20260924 21:14:40.877825 19917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:14:40.878171 19917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:14:40.879370 19917 hybrid_clock.cc:648] HybridClock initialized: now 1790284480879369 us; error 66 us; skew 500 ppm
Sep 24 21:14:40 dist-test-slave-q276 krb5kdc[18721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284480, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:14:40.882624 19917 init.cc:377] Logged in from keytab as kudu/127.6.210.130@KRBTEST.COM (short username kudu)
I20260924 21:14:40.883093 19917 webserver.cc:460] Webserver started at http://127.6.210.130:46661/ using document root <none> and password file <none>
I20260924 21:14:40.883378 19917 fs_manager.cc:359] Metadata directory not provided
I20260924 21:14:40.883464 19917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 24 21:14:40 dist-test-slave-q276 krb5kdc[18721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284480, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for HTTP/127.6.210.148@KRBTEST.COM
I20260924 21:14:40.904156 19917 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:14:40.905311 19935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:40.905586 19917 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:14:40.905694 19917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-1/wal
uuid: "05d698e68b6c4fcf834ab6dc64702f85"
format_stamp: "Formatted at 2026-09-24 21:14:39 on dist-test-slave-q276"
server_key: "7b1610be74e4b1fef643a119bd1caa6f"
server_key_iv: "ef05818642754d4de70457d01b1d7021"
server_key_version: "kuduclusterkey@0"
I20260924 21:14:40.906207 19917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:14:40.924852 19917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:14:40.927521 19917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:14:40.927690 19917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:14:40.927940 19917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:14:40.928330 19917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:14:40.928375 19917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:40.928406 19917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:14:40.928463 19917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:40.935948 19917 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.130:43753
I20260924 21:14:40.936055 20048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.130:43753 every 8 connection(s)
I20260924 21:14:40.936457 19917 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
I20260924 21:14:40.937180 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 19917
I20260924 21:14:40.937470 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.210.148:57969/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.131:39001
--local_ip_for_outbound_sockets=127.6.210.131
--tserver_master_addrs=127.6.210.190:42429
--webserver_port=35325
--webserver_interface=127.6.210.131
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:33871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.210.148:57969/kms with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 21:14:40 dist-test-slave-q276 krb5kdc[18721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284480, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:14:40.950779 19919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:40.937322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:53689 (local address 127.6.210.190:42429)
0924 21:14:40.937371 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:40.937373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:40.942087 (+ 4714us) server_negotiation.cc:407] Connection header received
0924 21:14:40.942693 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:40.942695 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:40.942710 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:40.942767 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:40.943359 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:40.943882 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:40.944891 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:40.945093 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:40.945179 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:40.947675 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:40.947677 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:40.947678 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:40.947686 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:40.949189 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:40.949622 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:40.949623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:40.949624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:40.949686 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:40.949963 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:40.949964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:40.949965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:40.950121 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:40.950195 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:40.950584 (+ 389us) server_negotiation.cc:299] Negotiation successful
0924 21:14:40.950676 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 21:14:40.951495 20051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:40.937331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:42429 (local address 127.6.210.130:53689)
0924 21:14:40.941873 (+ 4542us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:40.941954 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:40.942573 (+ 619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:40.942853 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:40.942857 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:40.943053 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:14:40.943252 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:40.943256 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:40.944001 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:40.944002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:40.944794 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:40.944796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:40.945208 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:40.945209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:40.945327 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:40.945895 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:14:40.945904 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:14:40.947568 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:14:40.949292 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:40.949300 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:40.949303 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:40.949544 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:40.949753 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:40.949754 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:40.949754 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:40.949877 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:40.950198 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:14:40.950205 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:14:40.950419 (+ 214us) client_negotiation.cc:769] Sending connection context
0924 21:14:40.951176 (+ 757us) client_negotiation.cc:240] Negotiation successful
0924 21:14:40.951328 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4492,"thread_start_us":97,"threads_started":1}
I20260924 21:14:40.951994 20049 heartbeater.cc:347] Connected to a master server at 127.6.210.190:42429
I20260924 21:14:40.952127 20049 heartbeater.cc:464] Registering TS with master...
I20260924 21:14:40.952329 20049 heartbeater.cc:511] Master 127.6.210.190:42429 requested a full tablet report, sending...
W20260924 21:14:40.953017 20049 heartbeater.cc:644] Failed to heartbeat to 127.6.210.190:42429 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260924 21:14:40.953089 20049 heartbeater.cc:464] Registering TS with master...
I20260924 21:14:40.953151 20049 heartbeater.cc:511] Master 127.6.210.190:42429 requested a full tablet report, sending...
I20260924 21:14:40.953505 20049 heartbeater.cc:464] Registering TS with master...
I20260924 21:14:40.953588 20049 heartbeater.cc:511] Master 127.6.210.190:42429 requested a full tablet report, sending...
W20260924 21:14:40.953920 20049 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 21:14:41.060283 20052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:14:41.060607 20052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:14:41.060657 20052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:14:41.062611 20052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:14:41.062685 20052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:14:41.062732 20052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.131
I20260924 21:14:41.065181 20052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:33871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.210.148:57969/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.131:39001
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.210.131
--webserver_port=35325
--tserver_master_addrs=127.6.210.190:42429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.131
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:14:41.065719 20052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:14:41.066092 20052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:14:41.076347 20059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:41.076301 20061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:41.076301 20058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:14:41.076604 20052 server_base.cc:1034] running on GCE node
I20260924 21:14:41.076865 20052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:14:41.077175 20052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:14:41.078343 20052 hybrid_clock.cc:648] HybridClock initialized: now 1790284481078318 us; error 37 us; skew 500 ppm
Sep 24 21:14:41 dist-test-slave-q276 krb5kdc[18721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284481, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:14:41.081540 20052 init.cc:377] Logged in from keytab as kudu/127.6.210.131@KRBTEST.COM (short username kudu)
I20260924 21:14:41.082145 20052 webserver.cc:460] Webserver started at http://127.6.210.131:35325/ using document root <none> and password file <none>
I20260924 21:14:41.082453 20052 fs_manager.cc:359] Metadata directory not provided
I20260924 21:14:41.082531 20052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 24 21:14:41 dist-test-slave-q276 krb5kdc[18721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284481, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for HTTP/127.6.210.148@KRBTEST.COM
I20260924 21:14:41.106514 20052 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:14:41.107511 20069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:41.107740 20052 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:14:41.107849 20052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-2/wal
uuid: "9405156f66fc40b99437f80bcb102367"
format_stamp: "Formatted at 2026-09-24 21:14:40 on dist-test-slave-q276"
server_key: "e05d2f133ad02fc2bc92b2609da73c12"
server_key_iv: "f4090d199ebf3570e0e95ad935b166ae"
server_key_version: "kuduclusterkey@0"
I20260924 21:14:41.108351 20052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:14:41.131565 20052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:14:41.133752 20052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:14:41.133912 20052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:14:41.134173 20052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:14:41.134599 20052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:14:41.134649 20052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:41.134680 20052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:14:41.134744 20052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:41.142165 20052 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.131:39001
I20260924 21:14:41.142640 20052 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
I20260924 21:14:41.143196 20182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.131:39001 every 8 connection(s)
I20260924 21:14:41.144706 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 20052
Sep 24 21:14:41 dist-test-slave-q276 krb5kdc[18721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284481, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:14:41.153800 19919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:41.143555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:49617 (local address 127.6.210.190:42429)
0924 21:14:41.143600 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:41.143602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:41.143851 (+ 249us) server_negotiation.cc:407] Connection header received
0924 21:14:41.144372 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:41.144374 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:41.144385 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:41.144444 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:41.145177 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:41.145742 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:41.146947 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:41.147257 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:41.147322 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:41.149684 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:41.149685 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:41.149686 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:41.149694 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:41.151573 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:41.152048 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:41.152049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:41.152050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:41.152110 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:41.152699 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:41.152700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:41.152700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:41.152838 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:41.153154 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:41.153563 (+ 409us) server_negotiation.cc:299] Negotiation successful
0924 21:14:41.153702 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 21:14:41.153937 20185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:41.143560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:42429 (local address 127.6.210.131:49617)
0924 21:14:41.143720 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:41.143756 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:41.144260 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:41.144514 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:41.144524 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:41.144698 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:14:41.145011 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:41.145018 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:41.145838 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:41.145839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:41.146846 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:41.146848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:41.147408 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:41.147410 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:41.147557 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:41.148198 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:14:41.148209 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:14:41.149590 (+ 1381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:14:41.151674 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:41.151681 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:41.151684 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:41.151989 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:41.152181 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:41.152182 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:41.152183 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:41.152616 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:41.153187 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:14:41.153193 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:14:41.153373 (+ 180us) client_negotiation.cc:769] Sending connection context
0924 21:14:41.153497 (+ 124us) client_negotiation.cc:240] Negotiation successful
0924 21:14:41.153662 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
I20260924 21:14:41.154726 20183 heartbeater.cc:347] Connected to a master server at 127.6.210.190:42429
I20260924 21:14:41.154831 20183 heartbeater.cc:464] Registering TS with master...
I20260924 21:14:41.155030 20183 heartbeater.cc:511] Master 127.6.210.190:42429 requested a full tablet report, sending...
W20260924 21:14:41.155782 20183 heartbeater.cc:644] Failed to heartbeat to 127.6.210.190:42429 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260924 21:14:41.155874 20183 heartbeater.cc:464] Registering TS with master...
I20260924 21:14:41.155939 20183 heartbeater.cc:511] Master 127.6.210.190:42429 requested a full tablet report, sending...
I20260924 21:14:41.156276 20183 heartbeater.cc:464] Registering TS with master...
I20260924 21:14:41.156369 20183 heartbeater.cc:511] Master 127.6.210.190:42429 requested a full tablet report, sending...
I20260924 21:14:41.156607 20187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:41.145171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50788 (local address 127.6.210.190:42429)
0924 21:14:41.145293 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:41.145295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:41.147106 (+ 1811us) server_negotiation.cc:407] Connection header received
0924 21:14:41.147164 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:41.147166 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:41.147180 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:41.147259 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:41.148041 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:41.148769 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:41.149817 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:41.150093 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:41.150173 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:41.152075 (+ 1902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:41.152076 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:41.152076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:41.152083 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:41.153850 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:41.154375 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:41.154376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:41.154376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:41.154464 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:41.155251 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:41.155252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:41.155253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:41.155419 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:41.156125 (+ 706us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:41.156167 (+ 42us) server_negotiation.cc:299] Negotiation successful
0924 21:14:41.156510 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":78,"threads_started":1}
W20260924 21:14:41.156860 20183 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 24 21:14:41 dist-test-slave-q276 krb5kdc[18721](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790284481, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:14:41.777099 19915 heartbeater.cc:464] Registering TS with master...
I20260924 21:14:41.777297 19915 heartbeater.cc:511] Master 127.6.210.190:42429 requested a full tablet report, sending...
I20260924 21:14:41.777733 19717 ts_manager.cc:194] Registered new tserver with Master: b3349cb471c04bb1835c530c725612b3 (127.6.210.129:35707)
I20260924 21:14:41.778512 19717 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.129@KRBTEST.COM'} at 127.6.210.129:46281
I20260924 21:14:41.954142 20049 heartbeater.cc:464] Registering TS with master...
I20260924 21:14:41.954324 20049 heartbeater.cc:511] Master 127.6.210.190:42429 requested a full tablet report, sending...
I20260924 21:14:41.955019 19716 ts_manager.cc:194] Registered new tserver with Master: 05d698e68b6c4fcf834ab6dc64702f85 (127.6.210.130:43753)
I20260924 21:14:41.955586 19716 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.130@KRBTEST.COM'} at 127.6.210.130:53689
Sep 24 21:14:42 dist-test-slave-q276 krb5kdc[18721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284481, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for HTTP/127.6.210.148@KRBTEST.COM
I20260924 21:14:42.157033 20183 heartbeater.cc:464] Registering TS with master...
I20260924 21:14:42.157179 20183 heartbeater.cc:511] Master 127.6.210.190:42429 requested a full tablet report, sending...
I20260924 21:14:42.157559 19716 ts_manager.cc:194] Registered new tserver with Master: 9405156f66fc40b99437f80bcb102367 (127.6.210.131:39001)
I20260924 21:14:42.158073 19716 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.131@KRBTEST.COM'} at 127.6.210.131:49617
I20260924 21:14:42.165627 6986 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:14:42.174490 20205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:42.166595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50790 (local address 127.6.210.190:42429)
0924 21:14:42.166883 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:42.166885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:42.166892 (+ 7us) server_negotiation.cc:407] Connection header received
0924 21:14:42.166912 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:42.166913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:42.166924 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:42.166989 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:42.167707 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:42.168495 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:42.169440 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:42.169756 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:42.169823 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:42.171365 (+ 1542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:42.171366 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:42.171367 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:42.171374 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:42.173205 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:42.173607 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:42.173608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:42.173608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:42.173669 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:42.173888 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:42.173888 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:42.173889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:42.174020 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:42.174116 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:42.174287 (+ 171us) server_negotiation.cc:299] Negotiation successful
0924 21:14:42.174373 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":59,"threads_started":1}
I20260924 21:14:42.176824 19716 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50790:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:14:42.179224 19716 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:14:42.191622 20211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:42.182796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:43753 (local address 127.0.0.1:38140)
0924 21:14:42.186397 (+ 3601us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:42.186449 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:42.186523 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:42.187024 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:42.187027 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:42.187030 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:14:42.187127 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:42.187130 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:42.188724 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:42.188725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:42.190179 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:42.190182 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:42.191285 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:42.191286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:42.191372 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:42.191389 (+ 17us) client_negotiation.cc:769] Sending connection context
0924 21:14:42.191441 (+ 52us) client_negotiation.cc:240] Negotiation successful
0924 21:14:42.191514 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3576,"mutex_wait_us":53,"thread_start_us":111,"threads_started":1}
I20260924 21:14:42.192320 20215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:42.183225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38140 (local address 127.6.210.130:43753)
0924 21:14:42.186832 (+ 3607us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:42.186839 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:42.186850 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:14:42.186889 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:42.186892 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:42.186911 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:42.187215 (+ 304us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:14:42.187299 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:42.188607 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:42.190315 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:42.191174 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:42.192121 (+ 947us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:42.192183 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:42.192208 (+ 25us) server_negotiation.cc:299] Negotiation successful
0924 21:14:42.192246 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3566,"thread_start_us":60,"threads_started":1}
I20260924 21:14:42.192889 20210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:42.182794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:35707 (local address 127.0.0.1:45966)
0924 21:14:42.186638 (+ 3844us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:42.186649 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:42.186698 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:42.187035 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:42.187036 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:42.187038 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:14:42.187125 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:42.187127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:42.188716 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:42.188717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:42.190154 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:42.190157 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:42.192660 (+ 2503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:42.192661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:42.192724 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:42.192728 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:14:42.192773 (+ 45us) client_negotiation.cc:240] Negotiation successful
0924 21:14:42.192811 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3826,"mutex_wait_us":132,"thread_start_us":103,"threads_started":1}
I20260924 21:14:42.193235 20213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:42.183080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45966 (local address 127.6.210.129:35707)
0924 21:14:42.183220 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:42.183225 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:42.186818 (+ 3593us) server_negotiation.cc:407] Connection header received
0924 21:14:42.186855 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:42.186858 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:42.186872 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:42.186939 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:14:42.187203 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:42.188621 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:42.190250 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:42.191340 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:42.191413 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:42.191482 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:42.193116 (+ 1634us) server_negotiation.cc:299] Negotiation successful
0924 21:14:42.193159 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":59,"threads_started":1}
I20260924 21:14:42.193172 19983 tablet_service.cc:1467] Processing CreateTablet for tablet 5a30e54d3ce54db6b829f2812c9d28f6 (DEFAULT_TABLE table=test-table [id=9fb822d4c91b46f9b02cf626fa25dc94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:14:42.193544 19983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a30e54d3ce54db6b829f2812c9d28f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:14:42.194090 19849 tablet_service.cc:1467] Processing CreateTablet for tablet 5a30e54d3ce54db6b829f2812c9d28f6 (DEFAULT_TABLE table=test-table [id=9fb822d4c91b46f9b02cf626fa25dc94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:14:42.194399 19849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a30e54d3ce54db6b829f2812c9d28f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:14:42.200495 20212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:42.182794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:39001 (local address 127.0.0.1:60672)
0924 21:14:42.186390 (+ 3596us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:42.186446 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:42.186521 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:42.188953 (+ 2432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:42.188953 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:42.188956 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:14:42.188992 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:42.188993 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:42.197871 (+ 8878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:42.197872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:42.199152 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:42.199155 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:42.200237 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:42.200239 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:42.200312 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:42.200316 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:14:42.200377 (+ 61us) client_negotiation.cc:240] Negotiation successful
0924 21:14:42.200418 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3559,"thread_start_us":110,"threads_started":1}
I20260924 21:14:42.200546 20214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:42.183080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60672 (local address 127.6.210.131:39001)
0924 21:14:42.183316 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:42.183319 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:42.186610 (+ 3291us) server_negotiation.cc:407] Connection header received
0924 21:14:42.186650 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:42.186653 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:42.186670 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:42.186735 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:14:42.191523 (+ 4788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:42.192563 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:42.199267 (+ 6704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:42.200140 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:42.200225 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:42.200280 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:42.200410 (+ 130us) server_negotiation.cc:299] Negotiation successful
0924 21:14:42.200462 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":60,"threads_started":1}
I20260924 21:14:42.201035 20217 tablet_bootstrap.cc:492] T 5a30e54d3ce54db6b829f2812c9d28f6 P b3349cb471c04bb1835c530c725612b3: Bootstrap starting.
I20260924 21:14:42.201573 20117 tablet_service.cc:1467] Processing CreateTablet for tablet 5a30e54d3ce54db6b829f2812c9d28f6 (DEFAULT_TABLE table=test-table [id=9fb822d4c91b46f9b02cf626fa25dc94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:14:42.201783 20217 tablet_bootstrap.cc:654] T 5a30e54d3ce54db6b829f2812c9d28f6 P b3349cb471c04bb1835c530c725612b3: Neither blocks nor log segments found. Creating new log.
I20260924 21:14:42.201931 20117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a30e54d3ce54db6b829f2812c9d28f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:14:42.202178 20217 log.cc:824] T 5a30e54d3ce54db6b829f2812c9d28f6 P b3349cb471c04bb1835c530c725612b3: Log is configured to *not* fsync() on all Append() calls
I20260924 21:14:42.202540 20216 tablet_bootstrap.cc:492] T 5a30e54d3ce54db6b829f2812c9d28f6 P 05d698e68b6c4fcf834ab6dc64702f85: Bootstrap starting.
I20260924 21:14:42.203140 20216 tablet_bootstrap.cc:654] T 5a30e54d3ce54db6b829f2812c9d28f6 P 05d698e68b6c4fcf834ab6dc64702f85: Neither blocks nor log segments found. Creating new log.
I20260924 21:14:42.203441 20216 log.cc:824] T 5a30e54d3ce54db6b829f2812c9d28f6 P 05d698e68b6c4fcf834ab6dc64702f85: Log is configured to *not* fsync() on all Append() calls
I20260924 21:14:42.204195 20217 tablet_bootstrap.cc:492] T 5a30e54d3ce54db6b829f2812c9d28f6 P b3349cb471c04bb1835c530c725612b3: No bootstrap required, opened a new log
I20260924 21:14:42.204257 20217 ts_tablet_manager.cc:1397] T 5a30e54d3ce54db6b829f2812c9d28f6 P b3349cb471c04bb1835c530c725612b3: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260924 21:14:42.205614 20220 tablet_bootstrap.cc:492] T 5a30e54d3ce54db6b829f2812c9d28f6 P 9405156f66fc40b99437f80bcb102367: Bootstrap starting.
I20260924 21:14:42.206216 20220 tablet_bootstrap.cc:654] T 5a30e54d3ce54db6b829f2812c9d28f6 P 9405156f66fc40b99437f80bcb102367: Neither blocks nor log segments found. Creating new log.
I20260924 21:14:42.206604 20220 log.cc:824] T 5a30e54d3ce54db6b829f2812c9d28f6 P 9405156f66fc40b99437f80bcb102367: Log is configured to *not* fsync() on all Append() calls
I20260924 21:14:42.206904 20217 raft_consensus.cc:348] T 5a30e54d3ce54db6b829f2812c9d28f6 P b3349cb471c04bb1835c530c725612b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d698e68b6c4fcf834ab6dc64702f85" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 43753 } } peers { permanent_uuid: "9405156f66fc40b99437f80bcb102367" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 39001 } } peers { permanent_uuid: "b3349cb471c04bb1835c530c725612b3" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 35707 } }
I20260924 21:14:42.207162 20217 raft_consensus.cc:374] T 5a30e54d3ce54db6b829f2812c9d28f6 P b3349cb471c04bb1835c530c725612b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:14:42.207263 20217 raft_consensus.cc:729] T 5a30e54d3ce54db6b829f2812c9d28f6 P b3349cb471c04bb1835c530c725612b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3349cb471c04bb1835c530c725612b3, State: Initialized, Role: FOLLOWER
I20260924 21:14:42.207350 20216 tablet_bootstrap.cc:492] T 5a30e54d3ce54db6b829f2812c9d28f6 P 05d698e68b6c4fcf834ab6dc64702f85: No bootstrap required, opened a new log
I20260924 21:14:42.207432 20216 ts_tablet_manager.cc:1397] T 5a30e54d3ce54db6b829f2812c9d28f6 P 05d698e68b6c4fcf834ab6dc64702f85: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260924 21:14:42.207563 20217 consensus_queue.cc:260] T 5a30e54d3ce54db6b829f2812c9d28f6 P b3349cb471c04bb1835c530c725612b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d698e68b6c4fcf834ab6dc64702f85" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 43753 } } peers { permanent_uuid: "9405156f66fc40b99437f80bcb102367" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 39001 } } peers { permanent_uuid: "b3349cb471c04bb1835c530c725612b3" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 35707 } }
I20260924 21:14:42.207891 20217 ts_tablet_manager.cc:1428] T 5a30e54d3ce54db6b829f2812c9d28f6 P b3349cb471c04bb1835c530c725612b3: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260924 21:14:42.208039 19915 heartbeater.cc:503] Master 127.6.210.190:42429 was elected leader, sending a full tablet report...
I20260924 21:14:42.209738 20220 tablet_bootstrap.cc:492] T 5a30e54d3ce54db6b829f2812c9d28f6 P 9405156f66fc40b99437f80bcb102367: No bootstrap required, opened a new log
I20260924 21:14:42.209848 20220 ts_tablet_manager.cc:1397] T 5a30e54d3ce54db6b829f2812c9d28f6 P 9405156f66fc40b99437f80bcb102367: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260924 21:14:42.209966 20216 raft_consensus.cc:348] T 5a30e54d3ce54db6b829f2812c9d28f6 P 05d698e68b6c4fcf834ab6dc64702f85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d698e68b6c4fcf834ab6dc64702f85" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 43753 } } peers { permanent_uuid: "9405156f66fc40b99437f80bcb102367" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 39001 } } peers { permanent_uuid: "b3349cb471c04bb1835c530c725612b3" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 35707 } }
I20260924 21:14:42.210175 20216 raft_consensus.cc:374] T 5a30e54d3ce54db6b829f2812c9d28f6 P 05d698e68b6c4fcf834ab6dc64702f85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:14:42.210248 20216 raft_consensus.cc:729] T 5a30e54d3ce54db6b829f2812c9d28f6 P 05d698e68b6c4fcf834ab6dc64702f85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05d698e68b6c4fcf834ab6dc64702f85, State: Initialized, Role: FOLLOWER
I20260924 21:14:42.210583 20216 consensus_queue.cc:260] T 5a30e54d3ce54db6b829f2812c9d28f6 P 05d698e68b6c4fcf834ab6dc64702f85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d698e68b6c4fcf834ab6dc64702f85" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 43753 } } peers { permanent_uuid: "9405156f66fc40b99437f80bcb102367" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 39001 } } peers { permanent_uuid: "b3349cb471c04bb1835c530c725612b3" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 35707 } }
I20260924 21:14:42.210958 20216 ts_tablet_manager.cc:1428] T 5a30e54d3ce54db6b829f2812c9d28f6 P 05d698e68b6c4fcf834ab6dc64702f85: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260924 21:14:42.211123 20049 heartbeater.cc:503] Master 127.6.210.190:42429 was elected leader, sending a full tablet report...
I20260924 21:14:42.213063 20220 raft_consensus.cc:348] T 5a30e54d3ce54db6b829f2812c9d28f6 P 9405156f66fc40b99437f80bcb102367 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d698e68b6c4fcf834ab6dc64702f85" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 43753 } } peers { permanent_uuid: "9405156f66fc40b99437f80bcb102367" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 39001 } } peers { permanent_uuid: "b3349cb471c04bb1835c530c725612b3" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 35707 } }
I20260924 21:14:42.213272 20220 raft_consensus.cc:374] T 5a30e54d3ce54db6b829f2812c9d28f6 P 9405156f66fc40b99437f80bcb102367 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:14:42.213367 20220 raft_consensus.cc:729] T 5a30e54d3ce54db6b829f2812c9d28f6 P 9405156f66fc40b99437f80bcb102367 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9405156f66fc40b99437f80bcb102367, State: Initialized, Role: FOLLOWER
I20260924 21:14:42.213585 20220 consensus_queue.cc:260] T 5a30e54d3ce54db6b829f2812c9d28f6 P 9405156f66fc40b99437f80bcb102367 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d698e68b6c4fcf834ab6dc64702f85" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 43753 } } peers { permanent_uuid: "9405156f66fc40b99437f80bcb102367" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 39001 } } peers { permanent_uuid: "b3349cb471c04bb1835c530c725612b3" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 35707 } }
I20260924 21:14:42.213897 20220 ts_tablet_manager.cc:1428] T 5a30e54d3ce54db6b829f2812c9d28f6 P 9405156f66fc40b99437f80bcb102367: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260924 21:14:42.213934 20183 heartbeater.cc:503] Master 127.6.210.190:42429 was elected leader, sending a full tablet report...
W20260924 21:14:42.246544 19916 tablet.cc:2369] T 5a30e54d3ce54db6b829f2812c9d28f6 P b3349cb471c04bb1835c530c725612b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 24 21:14:42 dist-test-slave-q276 krb5kdc[18721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284481, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for HTTP/127.6.210.148@KRBTEST.COM
I20260924 21:14:42.345630 20223 raft_consensus.cc:482] T 5a30e54d3ce54db6b829f2812c9d28f6 P 05d698e68b6c4fcf834ab6dc64702f85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:14:42.345780 20223 raft_consensus.cc:504] T 5a30e54d3ce54db6b829f2812c9d28f6 P 05d698e68b6c4fcf834ab6dc64702f85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d698e68b6c4fcf834ab6dc64702f85" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 43753 } } peers { permanent_uuid: "9405156f66fc40b99437f80bcb102367" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 39001 } } peers { permanent_uuid: "b3349cb471c04bb1835c530c725612b3" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 35707 } }
I20260924 21:14:42.346202 20223 leader_election.cc:288] T 5a30e54d3ce54db6b829f2812c9d28f6 P 05d698e68b6c4fcf834ab6dc64702f85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9405156f66fc40b99437f80bcb102367 (127.6.210.131:39001), b3349cb471c04bb1835c530c725612b3 (127.6.210.129:35707)
I20260924 21:14:42.351280 20214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:42.347195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:36191 (local address 127.6.210.131:39001)
0924 21:14:42.347234 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:42.347236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:42.347975 (+ 739us) server_negotiation.cc:407] Connection header received
0924 21:14:42.348085 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:42.348087 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:42.348096 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:42.348125 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:14:42.348329 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:42.349076 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:42.350388 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:42.350902 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:42.350947 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:42.350955 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:42.351165 (+ 210us) server_negotiation.cc:299] Negotiation successful
0924 21:14:42.351207 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 21:14:42.351363 20213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:42.346839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:57831 (local address 127.6.210.129:35707)
0924 21:14:42.346882 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:42.346884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:42.347703 (+ 819us) server_negotiation.cc:407] Connection header received
0924 21:14:42.347821 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:42.347822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:42.347832 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:42.347863 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:14:42.348112 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:42.348769 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:42.350433 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:42.350961 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:42.351014 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:42.351022 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:42.351248 (+ 226us) server_negotiation.cc:299] Negotiation successful
0924 21:14:42.351295 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 21:14:42.351593 20226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:42.346505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:35707 (local address 127.6.210.130:57831)
0924 21:14:42.347620 (+ 1115us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:42.347634 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:42.347737 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:42.347940 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:42.347941 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:42.347943 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:14:42.348026 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:42.348029 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:42.348875 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:42.348876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:42.350333 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:42.350337 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:42.351083 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:42.351084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:42.351162 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:42.351166 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:14:42.351449 (+ 283us) client_negotiation.cc:240] Negotiation successful
0924 21:14:42.351509 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1091,"thread_start_us":85,"threads_started":1}
I20260924 21:14:42.351953 20227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:42.347011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:39001 (local address 127.6.210.130:36191)
0924 21:14:42.347896 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:42.347908 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:42.348007 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:42.348190 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:42.348191 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:42.348192 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:14:42.348258 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:42.348260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:42.349178 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:42.349179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:42.350290 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:42.350292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:42.351031 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:42.351032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:42.351092 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:42.351095 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:14:42.351838 (+ 743us) client_negotiation.cc:240] Negotiation successful
0924 21:14:42.351880 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":866,"thread_start_us":64,"threads_started":1}
I20260924 21:14:42.352077 19869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a30e54d3ce54db6b829f2812c9d28f6" candidate_uuid: "05d698e68b6c4fcf834ab6dc64702f85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3349cb471c04bb1835c530c725612b3" is_pre_election: true
I20260924 21:14:42.352257 19869 raft_consensus.cc:2454] T 5a30e54d3ce54db6b829f2812c9d28f6 P b3349cb471c04bb1835c530c725612b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05d698e68b6c4fcf834ab6dc64702f85 in term 0.
I20260924 21:14:42.352358 20137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a30e54d3ce54db6b829f2812c9d28f6" candidate_uuid: "05d698e68b6c4fcf834ab6dc64702f85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9405156f66fc40b99437f80bcb102367" is_pre_election: true
I20260924 21:14:42.352515 20137 raft_consensus.cc:2454] T 5a30e54d3ce54db6b829f2812c9d28f6 P 9405156f66fc40b99437f80bcb102367 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05d698e68b6c4fcf834ab6dc64702f85 in term 0.
I20260924 21:14:42.352674 19938 leader_election.cc:302] T 5a30e54d3ce54db6b829f2812c9d28f6 P 05d698e68b6c4fcf834ab6dc64702f85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05d698e68b6c4fcf834ab6dc64702f85, b3349cb471c04bb1835c530c725612b3; no voters:
I20260924 21:14:42.352787 20223 raft_consensus.cc:2781] T 5a30e54d3ce54db6b829f2812c9d28f6 P 05d698e68b6c4fcf834ab6dc64702f85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:14:42.352820 20223 raft_consensus.cc:482] T 5a30e54d3ce54db6b829f2812c9d28f6 P 05d698e68b6c4fcf834ab6dc64702f85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:14:42.352839 20223 raft_consensus.cc:3037] T 5a30e54d3ce54db6b829f2812c9d28f6 P 05d698e68b6c4fcf834ab6dc64702f85 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:14:42.353682 20223 raft_consensus.cc:504] T 5a30e54d3ce54db6b829f2812c9d28f6 P 05d698e68b6c4fcf834ab6dc64702f85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d698e68b6c4fcf834ab6dc64702f85" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 43753 } } peers { permanent_uuid: "9405156f66fc40b99437f80bcb102367" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 39001 } } peers { permanent_uuid: "b3349cb471c04bb1835c530c725612b3" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 35707 } }
I20260924 21:14:42.353824 20223 leader_election.cc:288] T 5a30e54d3ce54db6b829f2812c9d28f6 P 05d698e68b6c4fcf834ab6dc64702f85 [CANDIDATE]: Term 1 election: Requested vote from peers 9405156f66fc40b99437f80bcb102367 (127.6.210.131:39001), b3349cb471c04bb1835c530c725612b3 (127.6.210.129:35707)
I20260924 21:14:42.354203 20137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a30e54d3ce54db6b829f2812c9d28f6" candidate_uuid: "05d698e68b6c4fcf834ab6dc64702f85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9405156f66fc40b99437f80bcb102367"
I20260924 21:14:42.354297 20137 raft_consensus.cc:3037] T 5a30e54d3ce54db6b829f2812c9d28f6 P 9405156f66fc40b99437f80bcb102367 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:14:42.355114 20137 raft_consensus.cc:2454] T 5a30e54d3ce54db6b829f2812c9d28f6 P 9405156f66fc40b99437f80bcb102367 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05d698e68b6c4fcf834ab6dc64702f85 in term 1.
I20260924 21:14:42.355280 19869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a30e54d3ce54db6b829f2812c9d28f6" candidate_uuid: "05d698e68b6c4fcf834ab6dc64702f85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3349cb471c04bb1835c530c725612b3"
I20260924 21:14:42.355340 19939 leader_election.cc:302] T 5a30e54d3ce54db6b829f2812c9d28f6 P 05d698e68b6c4fcf834ab6dc64702f85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05d698e68b6c4fcf834ab6dc64702f85, 9405156f66fc40b99437f80bcb102367; no voters:
I20260924 21:14:42.355368 19869 raft_consensus.cc:3037] T 5a30e54d3ce54db6b829f2812c9d28f6 P b3349cb471c04bb1835c530c725612b3 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:14:42.356169 19869 raft_consensus.cc:2454] T 5a30e54d3ce54db6b829f2812c9d28f6 P b3349cb471c04bb1835c530c725612b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05d698e68b6c4fcf834ab6dc64702f85 in term 1.
I20260924 21:14:42.356377 20223 raft_consensus.cc:2781] T 5a30e54d3ce54db6b829f2812c9d28f6 P 05d698e68b6c4fcf834ab6dc64702f85 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:14:42.356544 20223 raft_consensus.cc:686] T 5a30e54d3ce54db6b829f2812c9d28f6 P 05d698e68b6c4fcf834ab6dc64702f85 [term 1 LEADER]: Becoming Leader. State: Replica: 05d698e68b6c4fcf834ab6dc64702f85, State: Running, Role: LEADER
I20260924 21:14:42.356631 20223 consensus_queue.cc:237] T 5a30e54d3ce54db6b829f2812c9d28f6 P 05d698e68b6c4fcf834ab6dc64702f85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d698e68b6c4fcf834ab6dc64702f85" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 43753 } } peers { permanent_uuid: "9405156f66fc40b99437f80bcb102367" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 39001 } } peers { permanent_uuid: "b3349cb471c04bb1835c530c725612b3" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 35707 } }
I20260924 21:14:42.357564 19716 catalog_manager.cc:5495] T 5a30e54d3ce54db6b829f2812c9d28f6 P 05d698e68b6c4fcf834ab6dc64702f85 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05d698e68b6c4fcf834ab6dc64702f85 (127.6.210.130). New cstate: current_term: 1 leader_uuid: "05d698e68b6c4fcf834ab6dc64702f85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05d698e68b6c4fcf834ab6dc64702f85" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 43753 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9405156f66fc40b99437f80bcb102367" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 39001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3349cb471c04bb1835c530c725612b3" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 35707 } health_report { overall_health: UNKNOWN } } }
I20260924 21:14:42.376713 20215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:42.373558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38150 (local address 127.6.210.130:43753)
0924 21:14:42.373606 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:42.373608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:42.373617 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:14:42.373663 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:42.373664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:42.373684 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:42.373788 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:14:42.373978 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:42.374732 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:42.375796 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:42.376088 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:42.376162 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:42.376334 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:14:42.376481 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:14:42.376584 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:42.376608 (+ 24us) server_negotiation.cc:299] Negotiation successful
0924 21:14:42.376633 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 21:14:42.380049 20137 raft_consensus.cc:1261] T 5a30e54d3ce54db6b829f2812c9d28f6 P 9405156f66fc40b99437f80bcb102367 [term 1 FOLLOWER]: Refusing update from remote peer 05d698e68b6c4fcf834ab6dc64702f85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:14:42.380049 19869 raft_consensus.cc:1261] T 5a30e54d3ce54db6b829f2812c9d28f6 P b3349cb471c04bb1835c530c725612b3 [term 1 FOLLOWER]: Refusing update from remote peer 05d698e68b6c4fcf834ab6dc64702f85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:14:42.380478 20223 consensus_queue.cc:1035] T 5a30e54d3ce54db6b829f2812c9d28f6 P 05d698e68b6c4fcf834ab6dc64702f85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3349cb471c04bb1835c530c725612b3" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 35707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:14:42.380658 20223 consensus_queue.cc:1035] T 5a30e54d3ce54db6b829f2812c9d28f6 P 05d698e68b6c4fcf834ab6dc64702f85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9405156f66fc40b99437f80bcb102367" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 39001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:14:42.395946 20233 mvcc.cc:205] Tried to move back new op lower bound from 7333005239821299712 to 7333005239733239808. Current Snapshot: MvccSnapshot[applied={T|T < 7333005239821299712}]
I20260924 21:14:42.396124 20234 mvcc.cc:205] Tried to move back new op lower bound from 7333005239821299712 to 7333005239733239808. Current Snapshot: MvccSnapshot[applied={T|T < 7333005239821299712}]
I20260924 21:14:42.396507 19716 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:14:42.396605 19716 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50790:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:14:42.397470 19716 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e8e01028296147919455f083c3e2d7e9: Sending DeleteTablet for 3 replicas of tablet 5a30e54d3ce54db6b829f2812c9d28f6
I20260924 21:14:42.397897 19849 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a30e54d3ce54db6b829f2812c9d28f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:14:42 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:45966
I20260924 21:14:42.398237 19983 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a30e54d3ce54db6b829f2812c9d28f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:14:42 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:38140
I20260924 21:14:42.398798 20117 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a30e54d3ce54db6b829f2812c9d28f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:14:42 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:60672
I20260924 21:14:42.399312 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 19755
I20260924 21:14:42.406378 20242 tablet_replica.cc:330] T 5a30e54d3ce54db6b829f2812c9d28f6 P 05d698e68b6c4fcf834ab6dc64702f85: stopping tablet replica
I20260924 21:14:42.406621 20242 raft_consensus.cc:2229] T 5a30e54d3ce54db6b829f2812c9d28f6 P 05d698e68b6c4fcf834ab6dc64702f85 [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:14:42.406872 20242 raft_consensus.cc:2258] T 5a30e54d3ce54db6b829f2812c9d28f6 P 05d698e68b6c4fcf834ab6dc64702f85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:14:42.407119 20243 tablet_replica.cc:330] T 5a30e54d3ce54db6b829f2812c9d28f6 P 9405156f66fc40b99437f80bcb102367: stopping tablet replica
I20260924 21:14:42.407701 20242 ts_tablet_manager.cc:1905] T 5a30e54d3ce54db6b829f2812c9d28f6 P 05d698e68b6c4fcf834ab6dc64702f85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:14:42.407893 20243 raft_consensus.cc:2229] T 5a30e54d3ce54db6b829f2812c9d28f6 P 9405156f66fc40b99437f80bcb102367 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:14:42.408032 20243 raft_consensus.cc:2258] T 5a30e54d3ce54db6b829f2812c9d28f6 P 9405156f66fc40b99437f80bcb102367 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:14:42.410128 20242 ts_tablet_manager.cc:1918] T 5a30e54d3ce54db6b829f2812c9d28f6 P 05d698e68b6c4fcf834ab6dc64702f85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:14:42.410284 20242 log.cc:1196] T 5a30e54d3ce54db6b829f2812c9d28f6 P 05d698e68b6c4fcf834ab6dc64702f85: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-1/wal/wals/5a30e54d3ce54db6b829f2812c9d28f6
I20260924 21:14:42.410624 20242 ts_tablet_manager.cc:1939] T 5a30e54d3ce54db6b829f2812c9d28f6 P 05d698e68b6c4fcf834ab6dc64702f85: Deleting consensus metadata
I20260924 21:14:42.411134 19702 catalog_manager.cc:4841] TS 05d698e68b6c4fcf834ab6dc64702f85 (127.6.210.130:43753): tablet 5a30e54d3ce54db6b829f2812c9d28f6 (table test-table [id=9fb822d4c91b46f9b02cf626fa25dc94]) successfully deleted
I20260924 21:14:42.414372 20243 ts_tablet_manager.cc:1905] T 5a30e54d3ce54db6b829f2812c9d28f6 P 9405156f66fc40b99437f80bcb102367: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:14:42.415982 20243 ts_tablet_manager.cc:1918] T 5a30e54d3ce54db6b829f2812c9d28f6 P 9405156f66fc40b99437f80bcb102367: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:14:42.417047 20243 log.cc:1196] T 5a30e54d3ce54db6b829f2812c9d28f6 P 9405156f66fc40b99437f80bcb102367: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284333163253-6986-0/minicluster-data/ts-2/wal/wals/5a30e54d3ce54db6b829f2812c9d28f6
I20260924 21:14:42.417392 20243 ts_tablet_manager.cc:1939] T 5a30e54d3ce54db6b829f2812c9d28f6 P 9405156f66fc40b99437f80bcb102367: Deleting consensus metadata
I20260924 21:14:42.417907 19704 catalog_manager.cc:4841] TS 9405156f66fc40b99437f80bcb102367 (127.6.210.131:39001): tablet 5a30e54d3ce54db6b829f2812c9d28f6 (table test-table [id=9fb822d4c91b46f9b02cf626fa25dc94]) successfully deleted
W20260924 21:14:42.422119 19703 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.210.129:35707 (error 108)
I20260924 21:14:42.422928 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 19917
I20260924 21:14:42.429088 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 20052
I20260924 21:14:42.436848 20212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:42.436680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:35707 (local address 127.0.0.1:45968)
0924 21:14:42.436736 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:42.436801 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.129:35707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260924 21:14:42.437120 19703 catalog_manager.cc:4584] TS b3349cb471c04bb1835c530c725612b3 (127.6.210.129:35707): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5a30e54d3ce54db6b829f2812c9d28f6: Network error: Client connection negotiation failed: client connection to 127.6.210.129:35707: connect: Connection refused (error 111)
I20260924 21:14:42.437587 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 19684
I20260924 21:14:42.443961 6986 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260924 21:14:42.788630 6986 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260924 21:14:42.788729 6986 mini_ranger.cc:66] Stopping Ranger...
I20260924 21:14:43.133319 6986 mini_ranger.cc:68] Stopped Ranger
2026-09-24 21:14:43.133 UTC [18751] LOG: received smart shutdown request
2026-09-24 21:14:43.137 UTC [18751] LOG: background worker "logical replication launcher" (PID 18760) exited with exit code 1
2026-09-24 21:14:43.138 UTC [18755] LOG: shutting down
2026-09-24 21:14:43.315 UTC [18751] LOG: database system is shut down
2026-09-24T21:14:43Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49524 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:14:43 dist-test-slave-q276 krb5kdc[20253](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:14:43 dist-test-slave-q276 krb5kdc[20253](info): set up 2 sockets
Sep 24 21:14:43 dist-test-slave-q276 krb5kdc[20253](info): commencing operation
krb5kdc: starting...
W20260924 21:14:44.486159 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:14:44 dist-test-slave-q276 krb5kdc[20253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:14:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:14:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.210.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
I20260924 21:14:44.604423 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:39465
--webserver_interface=127.6.210.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:36505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:39465,127.6.210.189:46869,127.6.210.188:37887
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:14:44.684536 20266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:14:44.684752 20266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:14:44.684774 20266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:14:44.685861 20266 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:14:44.685894 20266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:14:44.685907 20266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:14:44.685920 20266 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:14:44.685931 20266 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:14:44.687383 20266 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:36505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:39465,127.6.210.189:46869,127.6.210.188:37887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:39465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:14:44.687579 20266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:14:44.687744 20266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:14:44.692472 20272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:44.692472 20274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:44.692538 20271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:14:44.692652 20266 server_base.cc:1034] running on GCE node
I20260924 21:14:44.692874 20266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:14:44.693198 20266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:14:44.694450 20266 hybrid_clock.cc:648] HybridClock initialized: now 1790284484694431 us; error 41 us; skew 500 ppm
Sep 24 21:14:44 dist-test-slave-q276 krb5kdc[20253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284484, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:14:44.700623 20266 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:14:44.701120 20266 webserver.cc:460] Webserver started at http://127.6.210.190:35075/ using document root <none> and password file <none>
I20260924 21:14:44.701396 20266 fs_manager.cc:359] Metadata directory not provided
I20260924 21:14:44.701510 20266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:14:44.701661 20266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:14:44.702869 20266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "b3005780e1904c408c772b9c9108e004"
format_stamp: "Formatted at 2026-09-24 21:14:44 on dist-test-slave-q276"
server_key: "44585d6717ab4c0d0ff5d48ec49dde82"
server_key_iv: "ec832604f4c95b5ecb46703cf59fd583"
server_key_version: "encryptionkey@0"
I20260924 21:14:44.703260 20266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "b3005780e1904c408c772b9c9108e004"
format_stamp: "Formatted at 2026-09-24 21:14:44 on dist-test-slave-q276"
server_key: "44585d6717ab4c0d0ff5d48ec49dde82"
server_key_iv: "ec832604f4c95b5ecb46703cf59fd583"
server_key_version: "encryptionkey@0"
I20260924 21:14:44.704640 20266 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:14:44.705405 20281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:44.705695 20266 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:14:44.705803 20266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "b3005780e1904c408c772b9c9108e004"
format_stamp: "Formatted at 2026-09-24 21:14:44 on dist-test-slave-q276"
server_key: "44585d6717ab4c0d0ff5d48ec49dde82"
server_key_iv: "ec832604f4c95b5ecb46703cf59fd583"
server_key_version: "encryptionkey@0"
I20260924 21:14:44.705891 20266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:14:44.722802 20266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:14:44.724792 20266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:14:44.724911 20266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:14:44.729115 20266 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:39465
I20260924 21:14:44.729187 20333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:39465 every 8 connection(s)
I20260924 21:14:44.729547 20266 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:14:44.730510 20334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:14:44.732064 20334 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39465 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 21:14:44 dist-test-slave-q276 krb5kdc[20253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284484, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:14:44.739266 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 20266
I20260924 21:14:44.739392 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:14:44.739624 6986 external_mini_cluster.cc:1428] Setting key 6e72774d3d81662725dffea4eeb7f4a8
I20260924 21:14:44.742081 20336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:44.732769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36078 (local address 127.6.210.190:39465)
0924 21:14:44.732967 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:44.732975 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:44.733098 (+ 123us) server_negotiation.cc:407] Connection header received
0924 21:14:44.733677 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:44.733680 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:44.733699 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:44.733803 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:44.734384 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:44.735201 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:44.736083 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:44.736403 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:44.736531 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:44.738618 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:44.738625 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:44.738626 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:44.738632 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:44.740164 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:44.740559 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:44.740562 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:44.740562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:44.740669 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:44.741001 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:44.741002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:44.741002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:44.741304 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:44.741397 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:44.741629 (+ 232us) server_negotiation.cc:299] Negotiation successful
0924 21:14:44.741830 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":87,"server-negotiator.queue_time_us":147,"thread_start_us":96,"threads_started":1}
I20260924 21:14:44.742092 20337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:44.732761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39465 (local address 127.0.0.1:36078)
0924 21:14:44.732958 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:44.733000 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:44.733518 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:44.733819 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:44.733822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:44.734090 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:14:44.734237 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:44.734241 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:44.735311 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:44.735312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:44.735992 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:44.735994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:44.736505 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:44.736506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:44.736569 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:44.737117 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:14:44.737125 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:14:44.738526 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:14:44.740261 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:44.740267 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:44.740269 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:44.740476 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:44.740765 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:44.740767 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:44.740767 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:44.740900 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:44.741407 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:14:44.741415 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:14:44.741482 (+ 67us) client_negotiation.cc:769] Sending connection context
0924 21:14:44.741578 (+ 96us) client_negotiation.cc:240] Negotiation successful
0924 21:14:44.741725 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"spinlock_wait_cycles":7552,"thread_start_us":96,"threads_started":1}
I20260924 21:14:44.742928 20334 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.210.189" port: 46869 } has no permanent_uuid. Determining permanent_uuid...
I20260924 21:14:44.743335 20337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:44.743185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:46869 (local address 127.0.0.1:38006)
0924 21:14:44.743244 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:44.743302 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.189:46869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260924 21:14:44.743507 20285 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.210.189:46869: connect: Connection refused (error 111)
I20260924 21:14:44.744009 20337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:44.743872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:46869 (local address 127.0.0.1:38022)
0924 21:14:44.743933 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:44.743968 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.189:46869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260924 21:14:44.744131 20334 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.210.189:46869: Network error: Client connection negotiation failed: client connection to 127.6.210.189:46869: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.6.210.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.189.keytab.
Entry for principal kudu/127.6.210.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.189.keytab.
I20260924 21:14:44.764678 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.189:46869
--webserver_interface=127.6.210.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.189.keytab
--principal=kudu/127.6.210.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:36505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:39465,127.6.210.189:46869,127.6.210.188:37887
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:14:44.793386 20334 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.210.189" port: 46869 } attempt: 1
I20260924 21:14:44.793927 20337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:44.793773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:46869 (local address 127.0.0.1:38028)
0924 21:14:44.793844 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:44.793894 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.189:46869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260924 21:14:44.794387 20337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:44.794307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:46869 (local address 127.0.0.1:38040)
0924 21:14:44.794338 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:44.794366 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.189:46869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260924 21:14:44.794492 20334 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.210.189:46869: Network error: Client connection negotiation failed: client connection to 127.6.210.189:46869: connect: Connection refused (error 111)
W20260924 21:14:44.845410 20341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:14:44.845631 20341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:14:44.845654 20341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:14:44.846750 20341 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:14:44.846783 20341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:14:44.846797 20341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:14:44.846810 20341 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:14:44.846822 20341 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:14:44.848318 20341 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:36505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:39465,127.6.210.189:46869,127.6.210.188:37887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.189.keytab
--principal=kudu/127.6.210.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.189:46869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.210.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:14:44.848511 20341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:14:44.848681 20341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 21:14:44.853636 20341 server_base.cc:1034] running on GCE node
W20260924 21:14:44.853624 20347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:44.853608 20349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:44.853617 20346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:14:44.854127 20341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:14:44.854488 20341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:14:44.855640 20341 hybrid_clock.cc:648] HybridClock initialized: now 1790284484855631 us; error 31 us; skew 500 ppm
Sep 24 21:14:44 dist-test-slave-q276 krb5kdc[20253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284484, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:14:44.858090 20341 init.cc:377] Logged in from keytab as kudu/127.6.210.189@KRBTEST.COM (short username kudu)
I20260924 21:14:44.858574 20341 webserver.cc:460] Webserver started at http://127.6.210.189:45829/ using document root <none> and password file <none>
I20260924 21:14:44.858809 20341 fs_manager.cc:359] Metadata directory not provided
I20260924 21:14:44.858897 20341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:14:44.859061 20341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:14:44.860149 20341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-1/data/instance:
uuid: "393571f51bb44e7a869544a7f45f46d0"
format_stamp: "Formatted at 2026-09-24 21:14:44 on dist-test-slave-q276"
server_key: "6077135699abfa8c1ad0ecd1c736dbf1"
server_key_iv: "3e6553eb745af46fb4f00ba4047f8962"
server_key_version: "encryptionkey@0"
I20260924 21:14:44.860509 20341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-1/wal/instance:
uuid: "393571f51bb44e7a869544a7f45f46d0"
format_stamp: "Formatted at 2026-09-24 21:14:44 on dist-test-slave-q276"
server_key: "6077135699abfa8c1ad0ecd1c736dbf1"
server_key_iv: "3e6553eb745af46fb4f00ba4047f8962"
server_key_version: "encryptionkey@0"
I20260924 21:14:44.861792 20341 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:14:44.862617 20356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:44.862630 20334 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.210.189" port: 46869 } attempt: 2
I20260924 21:14:44.862850 20341 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:14:44.862933 20341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-1/wal
uuid: "393571f51bb44e7a869544a7f45f46d0"
format_stamp: "Formatted at 2026-09-24 21:14:44 on dist-test-slave-q276"
server_key: "6077135699abfa8c1ad0ecd1c736dbf1"
server_key_iv: "3e6553eb745af46fb4f00ba4047f8962"
server_key_version: "encryptionkey@0"
I20260924 21:14:44.862988 20341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:14:44.863060 20337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:44.862885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:46869 (local address 127.0.0.1:38046)
0924 21:14:44.862988 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:44.863018 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.189:46869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260924 21:14:44.863476 20337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:44.863372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:46869 (local address 127.0.0.1:38056)
0924 21:14:44.863426 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:44.863450 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.189:46869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260924 21:14:44.863608 20334 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.210.189:46869: Network error: Client connection negotiation failed: client connection to 127.6.210.189:46869: connect: Connection refused (error 111)
I20260924 21:14:44.883800 20341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:14:44.885741 20341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:14:44.885918 20341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:14:44.889777 20341 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.189:46869
I20260924 21:14:44.889818 20408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.189:46869 every 8 connection(s)
I20260924 21:14:44.890177 20341 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-1/data/info.pb
I20260924 21:14:44.891084 20409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:14:44.892604 20409 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39465 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 21:14:44 dist-test-slave-q276 krb5kdc[20253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284484, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.189@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:14:44.899565 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 20341
I20260924 21:14:44.899681 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-1/wal/instance
I20260924 21:14:44.899885 6986 external_mini_cluster.cc:1428] Setting key 4a5d397cb381d0a630fac6fbed1cf1db
I20260924 21:14:44.901801 20336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:44.893321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36094 (local address 127.6.210.190:39465)
0924 21:14:44.893397 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:44.893399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:44.893586 (+ 187us) server_negotiation.cc:407] Connection header received
0924 21:14:44.894051 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:44.894052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:44.894062 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:44.894123 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:44.894807 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:44.895346 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:44.896371 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:44.896575 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:44.896627 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:44.898721 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:44.898722 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:44.898722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:44.898729 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:44.900278 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:44.900780 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:44.900780 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:44.900781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:44.900832 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:44.901121 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:44.901121 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:44.901121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:44.901224 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:44.901303 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:44.901624 (+ 321us) server_negotiation.cc:299] Negotiation successful
0924 21:14:44.901707 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260924 21:14:44.902022 20411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:44.893280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39465 (local address 127.0.0.1:36094)
0924 21:14:44.893464 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:44.893509 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:44.893930 (+ 421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:44.894195 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:44.894198 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:44.894528 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:14:44.894720 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:44.894724 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:44.895505 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:44.895506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:44.896292 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:44.896294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:44.896648 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:44.896648 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:44.896748 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:44.897197 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:14:44.897205 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:14:44.898633 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:14:44.900357 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:44.900365 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:44.900368 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:44.900706 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:44.900895 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:44.900896 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:44.900896 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:44.901057 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:44.901300 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:14:44.901308 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:14:44.901488 (+ 180us) client_negotiation.cc:769] Sending connection context
0924 21:14:44.901604 (+ 116us) client_negotiation.cc:240] Negotiation successful
0924 21:14:44.901771 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":85,"threads_started":1}
I20260924 21:14:44.902712 20409 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.210.189" port: 46869 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 21:14:44 dist-test-slave-q276 krb5kdc[20253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284484, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.189@KRBTEST.COM for kudu/127.6.210.189@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.210.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.188@KRBTEST.COM" created.
I20260924 21:14:44.910733 20411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:44.903023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:46869 (local address 127.0.0.1:38064)
0924 21:14:44.903076 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:44.903089 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:44.903132 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:44.903368 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:44.903370 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:44.903537 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:14:44.903601 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:44.903603 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:44.904460 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:44.904461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:44.905383 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:44.905385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:44.905839 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:44.905841 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:44.905924 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:44.906383 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:14:44.906389 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:14:44.907675 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:14:44.909504 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:44.909504 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:44.909505 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:44.909712 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:44.909958 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:44.909959 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:44.909959 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:44.910013 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:44.910413 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:14:44.910414 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:14:44.910453 (+ 39us) client_negotiation.cc:769] Sending connection context
0924 21:14:44.910516 (+ 63us) client_negotiation.cc:240] Negotiation successful
0924 21:14:44.910618 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260924 21:14:44.910804 20413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:44.903059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38064 (local address 127.6.210.189:46869)
0924 21:14:44.903224 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:44.903229 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:44.903247 (+ 18us) server_negotiation.cc:407] Connection header received
0924 21:14:44.903275 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:44.903277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:44.903287 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:44.903346 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:44.903683 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:44.904367 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:44.905473 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:44.905757 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:44.905814 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:44.907771 (+ 1957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:44.907779 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:44.907779 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:44.907786 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:44.909403 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:44.909828 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:44.909831 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:44.909831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:44.909888 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:44.910098 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:44.910098 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:44.910099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:44.910336 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:44.910445 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:44.910556 (+ 111us) server_negotiation.cc:299] Negotiation successful
0924 21:14:44.910670 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":137,"thread_start_us":111,"threads_started":1}
I20260924 21:14:44.911346 20409 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.210.188" port: 37887 } has no permanent_uuid. Determining permanent_uuid...
I20260924 21:14:44.911702 20411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:44.911580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:37887 (local address 127.0.0.1:38394)
0924 21:14:44.911628 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:44.911672 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.188:37887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260924 21:14:44.911823 20360 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.210.188:37887: connect: Connection refused (error 111)
I20260924 21:14:44.912329 20411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:44.912208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:37887 (local address 127.0.0.1:38396)
0924 21:14:44.912258 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:44.912300 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.188:37887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260924 21:14:44.912449 20409 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.210.188:37887: Network error: Client connection negotiation failed: client connection to 127.6.210.188:37887: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.188.keytab.
Entry for principal kudu/127.6.210.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.188.keytab.
I20260924 21:14:44.925899 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.188:37887
--webserver_interface=127.6.210.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.188.keytab
--principal=kudu/127.6.210.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:36505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:39465,127.6.210.189:46869,127.6.210.188:37887
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:14:44.954768 20334 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.210.189" port: 46869 } attempt: 3
Sep 24 21:14:44 dist-test-slave-q276 krb5kdc[20253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284484, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for kudu/127.6.210.189@KRBTEST.COM
I20260924 21:14:44.961659 20409 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.210.188" port: 37887 } attempt: 1
I20260924 21:14:44.962059 20411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:44.961915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:37887 (local address 127.0.0.1:38398)
0924 21:14:44.961983 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:44.962018 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.188:37887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260924 21:14:44.962606 20411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:44.962437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:37887 (local address 127.0.0.1:38400)
0924 21:14:44.962541 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:44.962570 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.188:37887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260924 21:14:44.962831 20337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:44.955185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:46869 (local address 127.0.0.1:38078)
0924 21:14:44.955263 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:44.955298 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:44.955357 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:44.955581 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:44.955582 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:44.955733 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:14:44.955798 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:44.955799 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:44.956764 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:44.956765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:44.957285 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:44.957287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:44.957807 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:44.957808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:44.957882 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:44.958423 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:14:44.958428 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:14:44.959807 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:14:44.961603 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:44.961604 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:44.961605 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:44.961817 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:44.962094 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:44.962095 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:44.962095 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:44.962168 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:44.962498 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:14:44.962499 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:14:44.962554 (+ 55us) client_negotiation.cc:769] Sending connection context
0924 21:14:44.962629 (+ 75us) client_negotiation.cc:240] Negotiation successful
0924 21:14:44.962737 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
W20260924 21:14:44.962860 20409 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.210.188:37887: Network error: Client connection negotiation failed: client connection to 127.6.210.188:37887: connect: Connection refused (error 111)
I20260924 21:14:44.962839 20413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:44.955315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38078 (local address 127.6.210.189:46869)
0924 21:14:44.955387 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:44.955389 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:44.955402 (+ 13us) server_negotiation.cc:407] Connection header received
0924 21:14:44.955432 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:44.955433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:44.955445 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:44.955524 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:44.955888 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:44.956674 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:44.957374 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:44.957699 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:44.957775 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:44.959904 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:44.959906 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:44.959906 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:44.959915 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:44.961493 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:44.961919 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:44.961920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:44.961921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:44.962011 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:44.962243 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:44.962244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:44.962245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:44.962419 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:44.962504 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:44.962673 (+ 169us) server_negotiation.cc:299] Negotiation successful
0924 21:14:44.962756 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 21:14:44.963439 20334 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.210.188" port: 37887 } has no permanent_uuid. Determining permanent_uuid...
I20260924 21:14:44.963877 20337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:44.963775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:37887 (local address 127.0.0.1:38404)
0924 21:14:44.963824 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:44.963852 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.188:37887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260924 21:14:44.964218 20337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:44.964139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:37887 (local address 127.0.0.1:38412)
0924 21:14:44.964182 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:44.964204 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.188:37887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260924 21:14:44.964325 20334 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.210.188:37887: Network error: Client connection negotiation failed: client connection to 127.6.210.188:37887: connect: Connection refused (error 111)
I20260924 21:14:44.995472 20334 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.210.188" port: 37887 } attempt: 1
I20260924 21:14:44.995988 20337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:44.995856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:37887 (local address 127.0.0.1:38424)
0924 21:14:44.995921 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:44.995960 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.188:37887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260924 21:14:44.996371 20337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:44.996290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:37887 (local address 127.0.0.1:38434)
0924 21:14:44.996331 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:44.996355 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.188:37887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260924 21:14:44.996479 20334 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.210.188:37887: Network error: Client connection negotiation failed: client connection to 127.6.210.188:37887: connect: Connection refused (error 111)
W20260924 21:14:45.010381 20416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:14:45.010599 20416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:14:45.010623 20416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:14:45.011794 20416 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:14:45.011845 20416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:14:45.011859 20416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:14:45.011873 20416 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:14:45.011884 20416 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:14:45.013792 20416 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:36505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:39465,127.6.210.189:46869,127.6.210.188:37887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.188.keytab
--principal=kudu/127.6.210.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.188:37887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.210.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:14:45.014101 20416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:14:45.014379 20416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:14:45.019254 20422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:45.019254 20424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:45.019281 20421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:14:45.019452 20416 server_base.cc:1034] running on GCE node
I20260924 21:14:45.019868 20416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:14:45.020212 20416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:14:45.021397 20416 hybrid_clock.cc:648] HybridClock initialized: now 1790284485021390 us; error 39 us; skew 500 ppm
Sep 24 21:14:45 dist-test-slave-q276 krb5kdc[20253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284485, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:14:45.023931 20416 init.cc:377] Logged in from keytab as kudu/127.6.210.188@KRBTEST.COM (short username kudu)
I20260924 21:14:45.024452 20416 webserver.cc:460] Webserver started at http://127.6.210.188:38617/ using document root <none> and password file <none>
I20260924 21:14:45.024693 20416 fs_manager.cc:359] Metadata directory not provided
I20260924 21:14:45.024763 20416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:14:45.024900 20416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:14:45.025960 20416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-2/data/instance:
uuid: "d58497190ae24b968792b866e585e412"
format_stamp: "Formatted at 2026-09-24 21:14:45 on dist-test-slave-q276"
server_key: "3277ef23731f5965e059c0c175ee533b"
server_key_iv: "9cd256f669f9efe137293d71773ef4eb"
server_key_version: "encryptionkey@0"
I20260924 21:14:45.026396 20416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-2/wal/instance:
uuid: "d58497190ae24b968792b866e585e412"
format_stamp: "Formatted at 2026-09-24 21:14:45 on dist-test-slave-q276"
server_key: "3277ef23731f5965e059c0c175ee533b"
server_key_iv: "9cd256f669f9efe137293d71773ef4eb"
server_key_version: "encryptionkey@0"
I20260924 21:14:45.027690 20416 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:14:45.028398 20431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:45.028612 20416 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:14:45.028726 20416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-2/wal
uuid: "d58497190ae24b968792b866e585e412"
format_stamp: "Formatted at 2026-09-24 21:14:45 on dist-test-slave-q276"
server_key: "3277ef23731f5965e059c0c175ee533b"
server_key_iv: "9cd256f669f9efe137293d71773ef4eb"
server_key_version: "encryptionkey@0"
I20260924 21:14:45.028839 20416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:14:45.031008 20409 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.210.188" port: 37887 } attempt: 2
I20260924 21:14:45.031380 20411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:45.031259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:37887 (local address 127.0.0.1:38438)
0924 21:14:45.031317 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:45.031352 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.188:37887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260924 21:14:45.031713 20411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:45.031625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:37887 (local address 127.0.0.1:38446)
0924 21:14:45.031667 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:45.031689 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.188:37887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260924 21:14:45.031793 20409 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.210.188:37887: Network error: Client connection negotiation failed: client connection to 127.6.210.188:37887: connect: Connection refused (error 111)
I20260924 21:14:45.046959 20416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:14:45.048981 20416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:14:45.049177 20416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:14:45.053402 20416 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.188:37887
I20260924 21:14:45.053436 20483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.188:37887 every 8 connection(s)
I20260924 21:14:45.053828 20416 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-2/data/info.pb
I20260924 21:14:45.054744 20484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:14:45.056234 20484 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39465 } has no permanent_uuid. Determining permanent_uuid...
I20260924 21:14:45.060735 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 20416
I20260924 21:14:45.060843 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284333163253-6986-0/minicluster-data/master-2/wal/instance
I20260924 21:14:45.061053 6986 external_mini_cluster.cc:1428] Setting key 185dc5095935734fca73eaeb5fc47911
Sep 24 21:14:45 dist-test-slave-q276 krb5kdc[20253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284485, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.188@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:14:45.066359 20336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:45.056914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36104 (local address 127.6.210.190:39465)
0924 21:14:45.056989 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:45.056991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:45.057203 (+ 212us) server_negotiation.cc:407] Connection header received
0924 21:14:45.057642 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:45.057643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:45.057653 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:45.057711 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:45.058382 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:45.058844 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:45.059808 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:45.060034 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:45.060091 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:45.062695 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:45.062697 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:45.062697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:45.062705 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:45.064815 (+ 2110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:45.065210 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:45.065212 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:45.065213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:45.065273 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:45.065591 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:45.065593 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:45.065593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:45.065745 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:45.065799 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:45.066168 (+ 369us) server_negotiation.cc:299] Negotiation successful
0924 21:14:45.066236 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 21:14:45.066604 20486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:45.056904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:39465 (local address 127.0.0.1:36104)
0924 21:14:45.057072 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:45.057113 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:45.057561 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:45.057764 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:45.057767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:45.058017 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:14:45.058241 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:45.058245 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:45.058912 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:45.058913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:45.059727 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:45.059729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:45.060108 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:45.060109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:45.060214 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:45.060732 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:14:45.060743 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:14:45.062595 (+ 1852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:14:45.064895 (+ 2300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:45.064902 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:45.064904 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:45.065131 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:45.065345 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:45.065345 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:45.065346 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:45.065492 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:45.065814 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:14:45.065821 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:14:45.066015 (+ 194us) client_negotiation.cc:769] Sending connection context
0924 21:14:45.066148 (+ 133us) client_negotiation.cc:240] Negotiation successful
0924 21:14:45.066340 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":93,"threads_started":1}
Sep 24 21:14:45 dist-test-slave-q276 krb5kdc[20253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.188@KRBTEST.COM
I20260924 21:14:45.067302 20484 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.210.189" port: 46869 } has no permanent_uuid. Determining permanent_uuid...
I20260924 21:14:45.070770 20487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:45.061444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38452 (local address 127.6.210.188:37887)
0924 21:14:45.061610 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:45.061615 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:45.061715 (+ 100us) server_negotiation.cc:407] Connection header received
0924 21:14:45.061768 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:45.061770 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:45.061788 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:45.061853 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:45.062586 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:45.063475 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:45.064625 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:45.064916 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:45.065058 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:45.067725 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:45.067732 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:45.067732 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:45.067740 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:45.069481 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:45.069870 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:45.069873 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:45.069873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:45.069933 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:45.070113 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:45.070114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:45.070114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:45.070305 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:45.070426 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:45.070570 (+ 144us) server_negotiation.cc:299] Negotiation successful
0924 21:14:45.070653 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":107,"threads_started":1}
I20260924 21:14:45.071619 20334 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.210.188" port: 37887 } attempt: 2
Sep 24 21:14:45 dist-test-slave-q276 krb5kdc[20253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284485, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.188@KRBTEST.COM for kudu/127.6.210.189@KRBTEST.COM
I20260924 21:14:45.075806 20486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:45.067603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:46869 (local address 127.0.0.1:38088)
0924 21:14:45.067730 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:45.067742 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:45.067791 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:45.067923 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:45.067924 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:45.068119 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:14:45.068185 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:45.068187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:45.069168 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:45.069169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:45.069736 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:45.069739 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:45.070237 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:45.070238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:45.070365 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:45.070843 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:14:45.070848 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:14:45.072296 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:14:45.074733 (+ 2437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:45.074734 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:45.074734 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:45.074945 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:45.075200 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:45.075200 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:45.075201 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:45.075262 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:45.075547 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:14:45.075548 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:14:45.075593 (+ 45us) client_negotiation.cc:769] Sending connection context
0924 21:14:45.075665 (+ 72us) client_negotiation.cc:240] Negotiation successful
0924 21:14:45.075745 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260924 21:14:45.075919 20413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:45.067677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38088 (local address 127.6.210.189:46869)
0924 21:14:45.067737 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:45.067740 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:45.067772 (+ 32us) server_negotiation.cc:407] Connection header received
0924 21:14:45.067848 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:45.067849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:45.067861 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:45.067922 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:45.068290 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:45.069083 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:45.069821 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:45.070166 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:45.070236 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:45.072382 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:45.072384 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:45.072384 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:45.072391 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:45.074659 (+ 2268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:45.075057 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:45.075058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:45.075059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:45.075123 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:45.075333 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:45.075334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:45.075335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:45.075460 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:45.075519 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:45.075730 (+ 211us) server_negotiation.cc:299] Negotiation successful
0924 21:14:45.075807 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
Sep 24 21:14:45 dist-test-slave-q276 krb5kdc[20253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284484, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for kudu/127.6.210.188@KRBTEST.COM
I20260924 21:14:45.076512 20484 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.210.188" port: 37887 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 21:14:45 dist-test-slave-q276 krb5kdc[20253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:14:45.079528 20487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:45.072068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38456 (local address 127.6.210.188:37887)
0924 21:14:45.072101 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:45.072103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:45.072110 (+ 7us) server_negotiation.cc:407] Connection header received
0924 21:14:45.072127 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:45.072128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:45.072136 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:45.072209 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:45.072518 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:45.073221 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:45.074201 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:45.074554 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:45.074878 (+ 324us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:45.076511 (+ 1633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:45.076512 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:45.076513 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:45.076519 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:45.078250 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:45.078647 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:45.078648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:45.078649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:45.078704 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:45.078922 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:45.078923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:45.078923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:45.079049 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:45.079319 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:45.079359 (+ 40us) server_negotiation.cc:299] Negotiation successful
0924 21:14:45.079436 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 21:14:45.079561 20337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:45.071834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:37887 (local address 127.0.0.1:38456)
0924 21:14:45.071876 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:45.071885 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:45.071918 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:45.072227 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:45.072228 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:45.072400 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:14:45.072429 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:45.072430 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:45.073286 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:45.073287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:45.074087 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:45.074092 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:45.074650 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:45.074651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:45.074715 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:45.075095 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:14:45.075099 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:14:45.076430 (+ 1331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:14:45.078392 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:45.078393 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:45.078394 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:45.078563 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:45.078788 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:45.078789 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:45.078789 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:45.078847 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:45.079133 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:14:45.079133 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:14:45.079181 (+ 48us) client_negotiation.cc:769] Sending connection context
0924 21:14:45.079229 (+ 48us) client_negotiation.cc:240] Negotiation successful
0924 21:14:45.079466 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
Sep 24 21:14:45 dist-test-slave-q276 krb5kdc[20253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284485, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.188@KRBTEST.COM for kudu/127.6.210.188@KRBTEST.COM
I20260924 21:14:45.081182 20336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:45.071557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36114 (local address 127.6.210.190:39465)
0924 21:14:45.071620 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:45.071622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:45.071633 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:14:45.071658 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:45.071659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:45.071670 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:45.071729 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:45.072331 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:45.073005 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:45.073831 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:45.074040 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:45.074123 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:45.077437 (+ 3314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:45.077439 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:45.077439 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:45.077445 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:45.079714 (+ 2269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:45.080037 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:45.080038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:45.080038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:45.080090 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:45.080293 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:45.080294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:45.080294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:45.080421 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:45.080612 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:45.080982 (+ 370us) server_negotiation.cc:299] Negotiation successful
0924 21:14:45.081075 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 21:14:45.082475 20334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3005780e1904c408c772b9c9108e004: Bootstrap starting.
I20260924 21:14:45.083258 20334 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3005780e1904c408c772b9c9108e004: Neither blocks nor log segments found. Creating new log.
I20260924 21:14:45.083583 20334 log.cc:824] T 00000000000000000000000000000000 P b3005780e1904c408c772b9c9108e004: Log is configured to *not* fsync() on all Append() calls
I20260924 21:14:45.083925 20486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:45.076756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:37887 (local address 127.0.0.1:38458)
0924 21:14:45.076786 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:45.076795 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:45.076828 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:45.077055 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:45.077056 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:45.077197 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:14:45.077221 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:45.077222 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:45.078125 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:45.078125 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:45.078720 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:45.078722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:45.079122 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:45.079123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:45.079172 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:45.079550 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:14:45.079554 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:14:45.080965 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:14:45.082829 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:45.082830 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:45.082830 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:45.083041 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:45.083255 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:45.083256 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:45.083256 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:45.083309 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:45.083640 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:14:45.083641 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:14:45.083676 (+ 35us) client_negotiation.cc:769] Sending connection context
0924 21:14:45.083737 (+ 61us) client_negotiation.cc:240] Negotiation successful
0924 21:14:45.083828 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260924 21:14:45.084218 20489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:45.076807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38458 (local address 127.6.210.188:37887)
0924 21:14:45.076952 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:45.076954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:45.076959 (+ 5us) server_negotiation.cc:407] Connection header received
0924 21:14:45.076971 (+ 12us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:45.076972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:45.076979 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:45.077053 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:45.077303 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:45.078032 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:45.078817 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:45.079039 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:45.079122 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:45.081047 (+ 1925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:45.081048 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:45.081049 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:45.081055 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:45.082749 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:45.083118 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:45.083119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:45.083119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:45.083176 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:45.083374 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:45.083375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:45.083375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:45.083553 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:45.083617 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:45.084041 (+ 424us) server_negotiation.cc:299] Negotiation successful
0924 21:14:45.084122 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":95,"threads_started":1}
I20260924 21:14:45.084983 20334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3005780e1904c408c772b9c9108e004: No bootstrap required, opened a new log
I20260924 21:14:45.085645 20484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412: Bootstrap starting.
Sep 24 21:14:45 dist-test-slave-q276 krb5kdc[20253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.189@KRBTEST.COM
I20260924 21:14:45.086383 20484 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412: Neither blocks nor log segments found. Creating new log.
I20260924 21:14:45.086676 20484 log.cc:824] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412: Log is configured to *not* fsync() on all Append() calls
I20260924 21:14:45.087167 20334 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3005780e1904c408c772b9c9108e004 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3005780e1904c408c772b9c9108e004" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39465 } } peers { permanent_uuid: "393571f51bb44e7a869544a7f45f46d0" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 46869 } } peers { permanent_uuid: "d58497190ae24b968792b866e585e412" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 37887 } }
I20260924 21:14:45.087313 20334 raft_consensus.cc:374] T 00000000000000000000000000000000 P b3005780e1904c408c772b9c9108e004 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:14:45.087358 20334 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3005780e1904c408c772b9c9108e004 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3005780e1904c408c772b9c9108e004, State: Initialized, Role: FOLLOWER
I20260924 21:14:45.087484 20484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412: No bootstrap required, opened a new log
I20260924 21:14:45.087478 20334 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3005780e1904c408c772b9c9108e004 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3005780e1904c408c772b9c9108e004" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39465 } } peers { permanent_uuid: "393571f51bb44e7a869544a7f45f46d0" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 46869 } } peers { permanent_uuid: "d58497190ae24b968792b866e585e412" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 37887 } }
I20260924 21:14:45.087805 20334 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3005780e1904c408c772b9c9108e004 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:14:45.087787 20492 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3005780e1904c408c772b9c9108e004 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3005780e1904c408c772b9c9108e004" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39465 } } peers { permanent_uuid: "393571f51bb44e7a869544a7f45f46d0" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 46869 } } peers { permanent_uuid: "d58497190ae24b968792b866e585e412" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 37887 } } }
I20260924 21:14:45.087991 20492 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3005780e1904c408c772b9c9108e004 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:14:45.089408 20413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:45.081816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38104 (local address 127.6.210.189:46869)
0924 21:14:45.081853 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:45.081854 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:45.081864 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:14:45.081888 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:45.081889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:45.081899 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:45.081955 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:45.082514 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:45.083093 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:45.083721 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:45.083981 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:45.084051 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:45.086417 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:45.086418 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:45.086418 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:45.086426 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:45.088191 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:45.088577 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:45.088578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:45.088579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:45.088633 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:45.088841 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:45.088842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:45.088843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:45.088979 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:45.089184 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:45.089222 (+ 38us) server_negotiation.cc:299] Negotiation successful
0924 21:14:45.089292 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 21:14:45.089716 20484 raft_consensus.cc:348] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3005780e1904c408c772b9c9108e004" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39465 } } peers { permanent_uuid: "393571f51bb44e7a869544a7f45f46d0" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 46869 } } peers { permanent_uuid: "d58497190ae24b968792b866e585e412" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 37887 } }
I20260924 21:14:45.089859 20484 raft_consensus.cc:374] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:14:45.089900 20484 raft_consensus.cc:729] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d58497190ae24b968792b866e585e412, State: Initialized, Role: FOLLOWER
I20260924 21:14:45.090014 20484 consensus_queue.cc:260] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3005780e1904c408c772b9c9108e004" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39465 } } peers { permanent_uuid: "393571f51bb44e7a869544a7f45f46d0" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 46869 } } peers { permanent_uuid: "d58497190ae24b968792b866e585e412" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 37887 } }
W20260924 21:14:45.090327 20503 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b3005780e1904c408c772b9c9108e004: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260924 21:14:45.090369 20484 sys_catalog.cc:564] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412 [sys.catalog]: configured and running, proceeding with master startup.
W20260924 21:14:45.090384 20503 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260924 21:14:45.092157 20516 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 21:14:45.092208 20516 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 21:14:45.090794 20266 master_runner.cc:186] Error getting master registration for 127.6.210.189:46869: OK, instance_id { permanent_uuid: "393571f51bb44e7a869544a7f45f46d0" instance_seqno: 1790284484888670 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 21:14:45.093608 20266 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 21:14:45.094511 6986 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:14:45.095880 20505 sys_catalog.cc:455] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3005780e1904c408c772b9c9108e004" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39465 } } peers { permanent_uuid: "393571f51bb44e7a869544a7f45f46d0" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 46869 } } peers { permanent_uuid: "d58497190ae24b968792b866e585e412" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 37887 } } }
I20260924 21:14:45.096046 20505 sys_catalog.cc:458] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:14:45.094539 20416 master_runner.cc:186] Error getting master registration for 127.6.210.189:46869: OK, instance_id { permanent_uuid: "393571f51bb44e7a869544a7f45f46d0" instance_seqno: 1790284484888670 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 21:14:45.096747 20416 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 21:14:45.106438 20336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:45.098661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36128 (local address 127.6.210.190:39465)
0924 21:14:45.098706 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:45.098708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:45.098841 (+ 133us) server_negotiation.cc:407] Connection header received
0924 21:14:45.098946 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:45.098947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:45.098958 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:45.098998 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:45.099811 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:45.100579 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:45.101619 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:45.101912 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:45.101955 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:45.103613 (+ 1658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:45.103614 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:45.103615 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:45.103622 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:45.105209 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:45.105580 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:45.105581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:45.105581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:45.105631 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:45.105819 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:45.105819 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:45.105820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:45.105938 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:45.106020 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:45.106221 (+ 201us) server_negotiation.cc:299] Negotiation successful
0924 21:14:45.106342 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 21:14:45.107846 20413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:45.098843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38118 (local address 127.6.210.189:46869)
0924 21:14:45.098884 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:45.098886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:45.099417 (+ 531us) server_negotiation.cc:407] Connection header received
0924 21:14:45.099576 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:45.099577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:45.099588 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:45.099618 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:45.100384 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:45.101101 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:45.102084 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:45.102753 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:45.102799 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:45.104773 (+ 1974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:45.104775 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:45.104775 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:45.104782 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:45.106405 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:45.107024 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:45.107025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:45.107025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:45.107078 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:45.107266 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:45.107267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:45.107267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:45.107389 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:45.107468 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:45.107676 (+ 208us) server_negotiation.cc:299] Negotiation successful
0924 21:14:45.107754 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 21:14:45.114377 20489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:45.099117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38460 (local address 127.6.210.188:37887)
0924 21:14:45.099162 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:45.099164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:45.107151 (+ 7987us) server_negotiation.cc:407] Connection header received
0924 21:14:45.107243 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:45.107244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:45.107256 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:45.107284 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:45.107797 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:45.108568 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:45.109670 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:45.110006 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:45.110046 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:45.111490 (+ 1444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:45.111492 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:45.111492 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:45.111499 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:45.113068 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:45.113493 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:45.113494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:45.113494 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:45.113550 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:45.113742 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:45.113743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:45.113743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:45.113860 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:45.113950 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:45.114173 (+ 223us) server_negotiation.cc:299] Negotiation successful
0924 21:14:45.114255 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 21:14:45.119818 20505 raft_consensus.cc:482] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:14:45.119936 20505 raft_consensus.cc:504] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3005780e1904c408c772b9c9108e004" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39465 } } peers { permanent_uuid: "393571f51bb44e7a869544a7f45f46d0" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 46869 } } peers { permanent_uuid: "d58497190ae24b968792b866e585e412" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 37887 } }
I20260924 21:14:45.120198 20505 leader_election.cc:288] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3005780e1904c408c772b9c9108e004 (127.6.210.190:39465), 393571f51bb44e7a869544a7f45f46d0 (127.6.210.189:46869)
I20260924 21:14:45.120769 20308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d58497190ae24b968792b866e585e412" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3005780e1904c408c772b9c9108e004" is_pre_election: true
I20260924 21:14:45.120923 20308 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b3005780e1904c408c772b9c9108e004 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d58497190ae24b968792b866e585e412 in term 0.
I20260924 21:14:45.121389 20435 leader_election.cc:302] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3005780e1904c408c772b9c9108e004, d58497190ae24b968792b866e585e412; no voters:
I20260924 21:14:45.122067 20505 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:14:45.122138 20505 raft_consensus.cc:482] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:14:45.122164 20505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:14:45.122910 20409 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.210.188" port: 37887 } attempt: 3
I20260924 21:14:45.123106 20505 raft_consensus.cc:504] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3005780e1904c408c772b9c9108e004" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39465 } } peers { permanent_uuid: "393571f51bb44e7a869544a7f45f46d0" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 46869 } } peers { permanent_uuid: "d58497190ae24b968792b866e585e412" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 37887 } }
I20260924 21:14:45.123222 20505 leader_election.cc:288] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412 [CANDIDATE]: Term 1 election: Requested vote from peers b3005780e1904c408c772b9c9108e004 (127.6.210.190:39465), 393571f51bb44e7a869544a7f45f46d0 (127.6.210.189:46869)
I20260924 21:14:45.123392 20308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d58497190ae24b968792b866e585e412" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3005780e1904c408c772b9c9108e004"
I20260924 21:14:45.123482 20308 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3005780e1904c408c772b9c9108e004 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:14:45.124404 20308 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b3005780e1904c408c772b9c9108e004 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d58497190ae24b968792b866e585e412 in term 1.
I20260924 21:14:45.124724 20435 leader_election.cc:302] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3005780e1904c408c772b9c9108e004, d58497190ae24b968792b866e585e412; no voters:
I20260924 21:14:45.124898 20505 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:14:45.125075 20505 raft_consensus.cc:686] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412 [term 1 LEADER]: Becoming Leader. State: Replica: d58497190ae24b968792b866e585e412, State: Running, Role: LEADER
I20260924 21:14:45.125159 20505 consensus_queue.cc:237] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3005780e1904c408c772b9c9108e004" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39465 } } peers { permanent_uuid: "393571f51bb44e7a869544a7f45f46d0" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 46869 } } peers { permanent_uuid: "d58497190ae24b968792b866e585e412" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 37887 } }
I20260924 21:14:45.125514 20505 sys_catalog.cc:455] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d58497190ae24b968792b866e585e412. Latest consensus state: current_term: 1 leader_uuid: "d58497190ae24b968792b866e585e412" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3005780e1904c408c772b9c9108e004" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39465 } } peers { permanent_uuid: "393571f51bb44e7a869544a7f45f46d0" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 46869 } } peers { permanent_uuid: "d58497190ae24b968792b866e585e412" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 37887 } } }
I20260924 21:14:45.125589 20505 sys_catalog.cc:458] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412 [sys.catalog]: This master's current role is: LEADER
Sep 24 21:14:45 dist-test-slave-q276 krb5kdc[20253](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284484, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.189@KRBTEST.COM for kudu/127.6.210.188@KRBTEST.COM
I20260924 21:14:45.131515 20411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:45.123164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:37887 (local address 127.0.0.1:38464)
0924 21:14:45.123248 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:45.123259 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:45.123302 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:45.123533 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:45.123534 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:45.123704 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:14:45.123748 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:45.123750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:45.124649 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:45.124650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:45.125454 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:45.125456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:45.125949 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:45.125950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:45.126018 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:45.126492 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:14:45.126497 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:14:45.127986 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:14:45.130209 (+ 2223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:45.130210 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:45.130211 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:45.130445 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:45.130717 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:45.130718 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:45.130718 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:45.130774 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:45.131203 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:14:45.131204 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:14:45.131259 (+ 55us) client_negotiation.cc:769] Sending connection context
0924 21:14:45.131317 (+ 58us) client_negotiation.cc:240] Negotiation successful
0924 21:14:45.131415 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260924 21:14:45.132076 20489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:45.123375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38464 (local address 127.6.210.188:37887)
0924 21:14:45.123413 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:45.123415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:45.123422 (+ 7us) server_negotiation.cc:407] Connection header received
0924 21:14:45.123439 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:45.123440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:45.123448 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:45.123493 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:45.123822 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:45.124555 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:45.125550 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:45.125839 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:45.125897 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:45.128104 (+ 2207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:45.128106 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:45.128106 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:45.128113 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:45.130073 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:45.130534 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:45.130535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:45.130535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:45.130617 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:45.130850 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:45.130851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:45.130851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:45.131075 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:45.131700 (+ 625us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:45.131764 (+ 64us) server_negotiation.cc:299] Negotiation successful
0924 21:14:45.131886 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 21:14:45.133577 20409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 393571f51bb44e7a869544a7f45f46d0: Bootstrap starting.
I20260924 21:14:45.134338 20409 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 393571f51bb44e7a869544a7f45f46d0: Neither blocks nor log segments found. Creating new log.
I20260924 21:14:45.134366 20527 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:14:45.134583 20527 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:14:45.134652 20409 log.cc:824] T 00000000000000000000000000000000 P 393571f51bb44e7a869544a7f45f46d0: Log is configured to *not* fsync() on all Append() calls
I20260924 21:14:45.135548 20409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 393571f51bb44e7a869544a7f45f46d0: No bootstrap required, opened a new log
I20260924 21:14:45.136569 20308 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b3005780e1904c408c772b9c9108e004 [term 1 FOLLOWER]: Refusing update from remote peer d58497190ae24b968792b866e585e412: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:14:45.137044 20525 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3005780e1904c408c772b9c9108e004" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:14:45.137491 20409 raft_consensus.cc:348] T 00000000000000000000000000000000 P 393571f51bb44e7a869544a7f45f46d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3005780e1904c408c772b9c9108e004" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39465 } } peers { permanent_uuid: "393571f51bb44e7a869544a7f45f46d0" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 46869 } } peers { permanent_uuid: "d58497190ae24b968792b866e585e412" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 37887 } }
I20260924 21:14:45.137643 20409 raft_consensus.cc:374] T 00000000000000000000000000000000 P 393571f51bb44e7a869544a7f45f46d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:14:45.137677 20409 raft_consensus.cc:729] T 00000000000000000000000000000000 P 393571f51bb44e7a869544a7f45f46d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 393571f51bb44e7a869544a7f45f46d0, State: Initialized, Role: FOLLOWER
I20260924 21:14:45.137782 20409 consensus_queue.cc:260] T 00000000000000000000000000000000 P 393571f51bb44e7a869544a7f45f46d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3005780e1904c408c772b9c9108e004" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39465 } } peers { permanent_uuid: "393571f51bb44e7a869544a7f45f46d0" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 46869 } } peers { permanent_uuid: "d58497190ae24b968792b866e585e412" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 37887 } }
I20260924 21:14:45.138067 20409 sys_catalog.cc:564] T 00000000000000000000000000000000 P 393571f51bb44e7a869544a7f45f46d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:14:45.138479 20492 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3005780e1904c408c772b9c9108e004 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d58497190ae24b968792b866e585e412. Latest consensus state: current_term: 1 leader_uuid: "d58497190ae24b968792b866e585e412" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3005780e1904c408c772b9c9108e004" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39465 } } peers { permanent_uuid: "393571f51bb44e7a869544a7f45f46d0" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 46869 } } peers { permanent_uuid: "d58497190ae24b968792b866e585e412" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 37887 } } }
I20260924 21:14:45.138562 20492 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3005780e1904c408c772b9c9108e004 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:14:45.138805 20381 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 393571f51bb44e7a869544a7f45f46d0 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:14:45.139360 20383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d58497190ae24b968792b866e585e412" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "393571f51bb44e7a869544a7f45f46d0" is_pre_election: true
I20260924 21:14:45.139688 20381 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 393571f51bb44e7a869544a7f45f46d0 [term 1 FOLLOWER]: Refusing update from remote peer d58497190ae24b968792b866e585e412: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:14:45.139783 20382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d58497190ae24b968792b866e585e412" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "393571f51bb44e7a869544a7f45f46d0"
I20260924 21:14:45.140543 20382 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 393571f51bb44e7a869544a7f45f46d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d58497190ae24b968792b866e585e412 in term 1.
I20260924 21:14:45.140906 20525 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412 [LEADER]: Connected to new peer: Peer: permanent_uuid: "393571f51bb44e7a869544a7f45f46d0" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 46869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:14:45.143054 20525 sys_catalog.cc:455] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d58497190ae24b968792b866e585e412" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3005780e1904c408c772b9c9108e004" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39465 } } peers { permanent_uuid: "393571f51bb44e7a869544a7f45f46d0" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 46869 } } peers { permanent_uuid: "d58497190ae24b968792b866e585e412" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 37887 } } }
I20260924 21:14:45.143141 20525 sys_catalog.cc:458] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412 [sys.catalog]: This master's current role is: LEADER
I20260924 21:14:45.143666 20505 sys_catalog.cc:455] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d58497190ae24b968792b866e585e412" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3005780e1904c408c772b9c9108e004" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39465 } } peers { permanent_uuid: "393571f51bb44e7a869544a7f45f46d0" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 46869 } } peers { permanent_uuid: "d58497190ae24b968792b866e585e412" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 37887 } } }
I20260924 21:14:45.143739 20505 sys_catalog.cc:458] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412 [sys.catalog]: This master's current role is: LEADER
I20260924 21:14:45.143985 20529 mvcc.cc:205] Tried to move back new op lower bound from 7333005251115802624 to 7333005251073277952. Current Snapshot: MvccSnapshot[applied={T|T < 7333005251115802624}]
I20260924 21:14:45.144291 20492 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3005780e1904c408c772b9c9108e004 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d58497190ae24b968792b866e585e412" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3005780e1904c408c772b9c9108e004" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39465 } } peers { permanent_uuid: "393571f51bb44e7a869544a7f45f46d0" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 46869 } } peers { permanent_uuid: "d58497190ae24b968792b866e585e412" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 37887 } } }
I20260924 21:14:45.144364 20492 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3005780e1904c408c772b9c9108e004 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:14:45.144778 20544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 393571f51bb44e7a869544a7f45f46d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d58497190ae24b968792b866e585e412" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3005780e1904c408c772b9c9108e004" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39465 } } peers { permanent_uuid: "393571f51bb44e7a869544a7f45f46d0" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 46869 } } peers { permanent_uuid: "d58497190ae24b968792b866e585e412" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 37887 } } }
I20260924 21:14:45.144866 20544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 393571f51bb44e7a869544a7f45f46d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:14:45.144912 20544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 393571f51bb44e7a869544a7f45f46d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d58497190ae24b968792b866e585e412. Latest consensus state: current_term: 1 leader_uuid: "d58497190ae24b968792b866e585e412" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3005780e1904c408c772b9c9108e004" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39465 } } peers { permanent_uuid: "393571f51bb44e7a869544a7f45f46d0" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 46869 } } peers { permanent_uuid: "d58497190ae24b968792b866e585e412" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 37887 } } }
I20260924 21:14:45.145009 20544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 393571f51bb44e7a869544a7f45f46d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:14:45.145046 20544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 393571f51bb44e7a869544a7f45f46d0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d58497190ae24b968792b866e585e412" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3005780e1904c408c772b9c9108e004" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39465 } } peers { permanent_uuid: "393571f51bb44e7a869544a7f45f46d0" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 46869 } } peers { permanent_uuid: "d58497190ae24b968792b866e585e412" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 37887 } } }
I20260924 21:14:45.145090 20544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 393571f51bb44e7a869544a7f45f46d0 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:14:45.145325 20545 mvcc.cc:205] Tried to move back new op lower bound from 7333005251115802624 to 7333005251073277952. Current Snapshot: MvccSnapshot[applied={T|T < 7333005251115802624}]
I20260924 21:14:45.145423 20531 mvcc.cc:205] Tried to move back new op lower bound from 7333005251115802624 to 7333005251073277952. Current Snapshot: MvccSnapshot[applied={T|T < 7333005251115802624 or (T in {7333005251115802624})}]
I20260924 21:14:45.146685 20527 catalog_manager.cc:1347] Generated new cluster ID: 602544a0f18d4cfb8f439cfc33963510
I20260924 21:14:45.146744 20527 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:14:45.146958 20543 catalog_manager.cc:1259] Loaded cluster ID: 602544a0f18d4cfb8f439cfc33963510
I20260924 21:14:45.147008 20543 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 393571f51bb44e7a869544a7f45f46d0: loading cluster ID for follower catalog manager: success
W20260924 21:14:45.147070 20543 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 393571f51bb44e7a869544a7f45f46d0: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260924 21:14:45.147101 20543 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260924 21:14:45.164081 20527 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:14:45.164826 20527 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:14:45.183297 20527 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d58497190ae24b968792b866e585e412: Generated new TSK 0
I20260924 21:14:45.183517 20527 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:14:46.090790 20503 catalog_manager.cc:1259] Loaded cluster ID: 602544a0f18d4cfb8f439cfc33963510
I20260924 21:14:46.090859 20503 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b3005780e1904c408c772b9c9108e004: loading cluster ID for follower catalog manager: success
I20260924 21:14:46.091712 20503 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b3005780e1904c408c772b9c9108e004: acquiring CA information for follower catalog manager: success
I20260924 21:14:46.091984 20503 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b3005780e1904c408c772b9c9108e004: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 21:14:46.148239 20543 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 393571f51bb44e7a869544a7f45f46d0: acquiring CA information for follower catalog manager: success
I20260924 21:14:46.148717 20543 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 393571f51bb44e7a869544a7f45f46d0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 21:14:46.239929 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 20266
I20260924 21:14:46.245448 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 20341
I20260924 21:14:46.250906 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 20416
2026-09-24T21:14:46Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2816 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:14:46 dist-test-slave-q276 krb5kdc[20558](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:14:46 dist-test-slave-q276 krb5kdc[20558](info): set up 2 sockets
Sep 24 21:14:46 dist-test-slave-q276 krb5kdc[20558](info): commencing operation
krb5kdc: starting...
W20260924 21:14:47.302477 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:14:47 dist-test-slave-q276 krb5kdc[20558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284487, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:14:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:14:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.210.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
I20260924 21:14:47.420994 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:39831
--webserver_interface=127.6.210.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:41057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:39831
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:14:47.505528 20571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:14:47.505770 20571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:14:47.505810 20571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:14:47.507046 20571 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:14:47.507126 20571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:14:47.507153 20571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:14:47.507186 20571 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:14:47.507208 20571 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:14:47.508704 20571 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:41057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:39831
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:39831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:14:47.508992 20571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:14:47.509230 20571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:14:47.514209 20576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:47.514209 20579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:14:47.514238 20571 server_base.cc:1034] running on GCE node
W20260924 21:14:47.514213 20577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:14:47.514715 20571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:14:47.515071 20571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:14:47.516244 20571 hybrid_clock.cc:648] HybridClock initialized: now 1790284487516229 us; error 37 us; skew 500 ppm
Sep 24 21:14:47 dist-test-slave-q276 krb5kdc[20558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284487, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:14:47.518867 20571 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:14:47.519273 20571 webserver.cc:460] Webserver started at http://127.6.210.190:39871/ using document root <none> and password file <none>
I20260924 21:14:47.519480 20571 fs_manager.cc:359] Metadata directory not provided
I20260924 21:14:47.519522 20571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:14:47.519609 20571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:14:47.520632 20571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "39d5f69336004229bbb5239589262c86"
format_stamp: "Formatted at 2026-09-24 21:14:47 on dist-test-slave-q276"
server_key: "4938c4731910440ab7f36b9fd68864ab"
server_key_iv: "837d022864ed020b18a7477a70ab5a56"
server_key_version: "encryptionkey@0"
I20260924 21:14:47.520942 20571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "39d5f69336004229bbb5239589262c86"
format_stamp: "Formatted at 2026-09-24 21:14:47 on dist-test-slave-q276"
server_key: "4938c4731910440ab7f36b9fd68864ab"
server_key_iv: "837d022864ed020b18a7477a70ab5a56"
server_key_version: "encryptionkey@0"
I20260924 21:14:47.522214 20571 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:14:47.523054 20586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:47.523211 20571 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:14:47.523294 20571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "39d5f69336004229bbb5239589262c86"
format_stamp: "Formatted at 2026-09-24 21:14:47 on dist-test-slave-q276"
server_key: "4938c4731910440ab7f36b9fd68864ab"
server_key_iv: "837d022864ed020b18a7477a70ab5a56"
server_key_version: "encryptionkey@0"
I20260924 21:14:47.523350 20571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:14:47.536141 20571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:14:47.538103 20571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:14:47.538218 20571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:14:47.541997 20571 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:39831
I20260924 21:14:47.542095 20638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:39831 every 8 connection(s)
I20260924 21:14:47.542475 20571 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:14:47.543490 20639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:14:47.545791 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 20571
I20260924 21:14:47.545876 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:14:47.545939 20639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39d5f69336004229bbb5239589262c86: Bootstrap starting.
I20260924 21:14:47.546116 6986 external_mini_cluster.cc:1428] Setting key 6312ee59333a6e209dd941b5fca24e81
I20260924 21:14:47.546646 20639 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39d5f69336004229bbb5239589262c86: Neither blocks nor log segments found. Creating new log.
I20260924 21:14:47.547000 20639 log.cc:824] T 00000000000000000000000000000000 P 39d5f69336004229bbb5239589262c86: Log is configured to *not* fsync() on all Append() calls
I20260924 21:14:47.547741 20639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39d5f69336004229bbb5239589262c86: No bootstrap required, opened a new log
I20260924 21:14:47.549641 20639 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39d5f69336004229bbb5239589262c86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39d5f69336004229bbb5239589262c86" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39831 } }
I20260924 21:14:47.549808 20639 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39d5f69336004229bbb5239589262c86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:14:47.549852 20639 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39d5f69336004229bbb5239589262c86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39d5f69336004229bbb5239589262c86, State: Initialized, Role: FOLLOWER
I20260924 21:14:47.550042 20639 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39d5f69336004229bbb5239589262c86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39d5f69336004229bbb5239589262c86" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39831 } }
I20260924 21:14:47.550143 20639 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39d5f69336004229bbb5239589262c86 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:14:47.550189 20639 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39d5f69336004229bbb5239589262c86 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:14:47.550239 20639 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39d5f69336004229bbb5239589262c86 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:14:47.550997 20639 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39d5f69336004229bbb5239589262c86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39d5f69336004229bbb5239589262c86" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39831 } }
I20260924 21:14:47.551112 20639 leader_election.cc:302] T 00000000000000000000000000000000 P 39d5f69336004229bbb5239589262c86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39d5f69336004229bbb5239589262c86; no voters:
I20260924 21:14:47.551321 20639 leader_election.cc:288] T 00000000000000000000000000000000 P 39d5f69336004229bbb5239589262c86 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:14:47.551353 20644 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39d5f69336004229bbb5239589262c86 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:14:47.551601 20639 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39d5f69336004229bbb5239589262c86 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:14:47.551864 20644 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39d5f69336004229bbb5239589262c86 [term 1 LEADER]: Becoming Leader. State: Replica: 39d5f69336004229bbb5239589262c86, State: Running, Role: LEADER
I20260924 21:14:47.551990 20644 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39d5f69336004229bbb5239589262c86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39d5f69336004229bbb5239589262c86" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39831 } }
I20260924 21:14:47.552500 20646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39d5f69336004229bbb5239589262c86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39d5f69336004229bbb5239589262c86. Latest consensus state: current_term: 1 leader_uuid: "39d5f69336004229bbb5239589262c86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39d5f69336004229bbb5239589262c86" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39831 } } }
I20260924 21:14:47.552618 20646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39d5f69336004229bbb5239589262c86 [sys.catalog]: This master's current role is: LEADER
I20260924 21:14:47.552899 20656 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 24 21:14:47 dist-test-slave-q276 krb5kdc[20558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284487, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:14:47.553349 20645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39d5f69336004229bbb5239589262c86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39d5f69336004229bbb5239589262c86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39d5f69336004229bbb5239589262c86" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39831 } } }
I20260924 21:14:47.553872 20645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39d5f69336004229bbb5239589262c86 [sys.catalog]: This master's current role is: LEADER
I20260924 21:14:47.554165 20656 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:14:47.558009 20642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:47.546711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45692 (local address 127.6.210.190:39831)
0924 21:14:47.546917 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:47.546927 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:47.546960 (+ 33us) server_negotiation.cc:407] Connection header received
0924 21:14:47.547625 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:47.547634 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:47.547687 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:47.547846 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:47.548486 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:47.549382 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:47.550312 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:47.550944 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:47.551077 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:47.554512 (+ 3435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:47.554517 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:47.554517 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:47.554529 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:47.556325 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:47.556775 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:47.556778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:47.556778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:47.556877 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:47.557122 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:47.557123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:47.557123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:47.557350 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:47.557460 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:47.557628 (+ 168us) server_negotiation.cc:299] Negotiation successful
0924 21:14:47.557734 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":82,"threads_started":1}
I20260924 21:14:47.559051 6986 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:14:47.559350 20656 catalog_manager.cc:1347] Generated new cluster ID: 9e763e627b9a4cc0b9a2176517b675b2
I20260924 21:14:47.559407 20656 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:14:47.587072 20656 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:14:47.587638 20656 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:14:47.591941 20656 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39d5f69336004229bbb5239589262c86: Generated new TSK 0
I20260924 21:14:47.592165 20656 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:14:47.615514 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 20571
2026-09-24T21:14:47Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1362 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:14:47 dist-test-slave-q276 krb5kdc[20666](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:14:47 dist-test-slave-q276 krb5kdc[20666](info): set up 2 sockets
Sep 24 21:14:47 dist-test-slave-q276 krb5kdc[20666](info): commencing operation
krb5kdc: starting...
W20260924 21:14:48.654421 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:14:48 dist-test-slave-q276 krb5kdc[20666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284488, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:14:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:14:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.210.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
I20260924 21:14:48.772826 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:34097
--webserver_interface=127.6.210.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:43871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:34097
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:14:48.852373 20679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:14:48.852624 20679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:14:48.852663 20679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:14:48.854171 20679 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:14:48.854277 20679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:14:48.854336 20679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:14:48.854378 20679 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:14:48.854408 20679 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:14:48.856364 20679 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:43871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:34097
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:34097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:14:48.856680 20679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:14:48.856986 20679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:14:48.862172 20687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:48.862180 20685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:48.862354 20684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:14:48.862627 20679 server_base.cc:1034] running on GCE node
I20260924 21:14:48.862859 20679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:14:48.863144 20679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:14:48.864310 20679 hybrid_clock.cc:648] HybridClock initialized: now 1790284488864294 us; error 47 us; skew 500 ppm
Sep 24 21:14:48 dist-test-slave-q276 krb5kdc[20666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284488, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:14:48.867130 20679 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:14:48.867591 20679 webserver.cc:460] Webserver started at http://127.6.210.190:44911/ using document root <none> and password file <none>
I20260924 21:14:48.867790 20679 fs_manager.cc:359] Metadata directory not provided
I20260924 21:14:48.867830 20679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:14:48.867918 20679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:14:48.868918 20679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "e5faabaa1eee4d84a3504cde1f09eedb"
format_stamp: "Formatted at 2026-09-24 21:14:48 on dist-test-slave-q276"
server_key: "20b16c68c5ce2f11c4de44030da35020"
server_key_iv: "278bc3738ccdbfd86105d88b71a77ccd"
server_key_version: "encryptionkey@0"
I20260924 21:14:48.869222 20679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "e5faabaa1eee4d84a3504cde1f09eedb"
format_stamp: "Formatted at 2026-09-24 21:14:48 on dist-test-slave-q276"
server_key: "20b16c68c5ce2f11c4de44030da35020"
server_key_iv: "278bc3738ccdbfd86105d88b71a77ccd"
server_key_version: "encryptionkey@0"
I20260924 21:14:48.870471 20679 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 21:14:48.871138 20694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:48.871304 20679 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:48.871387 20679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "e5faabaa1eee4d84a3504cde1f09eedb"
format_stamp: "Formatted at 2026-09-24 21:14:48 on dist-test-slave-q276"
server_key: "20b16c68c5ce2f11c4de44030da35020"
server_key_iv: "278bc3738ccdbfd86105d88b71a77ccd"
server_key_version: "encryptionkey@0"
I20260924 21:14:48.871443 20679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:14:48.898139 20679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:14:48.900381 20679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:14:48.900494 20679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:14:48.904268 20679 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:34097
I20260924 21:14:48.904371 20746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:34097 every 8 connection(s)
I20260924 21:14:48.904681 20679 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:14:48.905696 20747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:14:48.907603 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 20679
I20260924 21:14:48.907708 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:14:48.907912 6986 external_mini_cluster.cc:1428] Setting key 0a9b4642efe4053beef46e2927897a0a
I20260924 21:14:48.908221 20747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb: Bootstrap starting.
I20260924 21:14:48.908874 20747 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb: Neither blocks nor log segments found. Creating new log.
I20260924 21:14:48.909157 20747 log.cc:824] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb: Log is configured to *not* fsync() on all Append() calls
I20260924 21:14:48.909837 20747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb: No bootstrap required, opened a new log
I20260924 21:14:48.911633 20747 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5faabaa1eee4d84a3504cde1f09eedb" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34097 } }
I20260924 21:14:48.911789 20747 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:14:48.911836 20747 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5faabaa1eee4d84a3504cde1f09eedb, State: Initialized, Role: FOLLOWER
I20260924 21:14:48.911989 20747 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5faabaa1eee4d84a3504cde1f09eedb" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34097 } }
I20260924 21:14:48.912091 20747 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:14:48.912137 20747 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:14:48.912185 20747 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:14:48.912732 20747 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5faabaa1eee4d84a3504cde1f09eedb" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34097 } }
I20260924 21:14:48.912842 20747 leader_election.cc:302] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5faabaa1eee4d84a3504cde1f09eedb; no voters:
I20260924 21:14:48.913007 20747 leader_election.cc:288] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:14:48.913031 20752 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:14:48.913141 20752 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [term 1 LEADER]: Becoming Leader. State: Replica: e5faabaa1eee4d84a3504cde1f09eedb, State: Running, Role: LEADER
I20260924 21:14:48.913270 20747 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:14:48.913285 20752 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5faabaa1eee4d84a3504cde1f09eedb" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34097 } }
I20260924 21:14:48.913719 20754 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5faabaa1eee4d84a3504cde1f09eedb. Latest consensus state: current_term: 1 leader_uuid: "e5faabaa1eee4d84a3504cde1f09eedb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5faabaa1eee4d84a3504cde1f09eedb" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34097 } } }
I20260924 21:14:48.913820 20754 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [sys.catalog]: This master's current role is: LEADER
I20260924 21:14:48.914024 20753 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5faabaa1eee4d84a3504cde1f09eedb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5faabaa1eee4d84a3504cde1f09eedb" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34097 } } }
I20260924 21:14:48.914330 20753 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [sys.catalog]: This master's current role is: LEADER
Sep 24 21:14:48 dist-test-slave-q276 krb5kdc[20666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284488, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
W20260924 21:14:48.915350 20768 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 21:14:48.915448 20768 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 21:14:48.915515 20759 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:14:48.915691 20759 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:14:48.917207 20759 catalog_manager.cc:1347] Generated new cluster ID: cf8ab8b0807d453abf05c7b382fd51e9
I20260924 21:14:48.917260 20759 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:14:48.920873 20750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:48.908426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46066 (local address 127.6.210.190:34097)
0924 21:14:48.908631 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:48.908640 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:48.908675 (+ 35us) server_negotiation.cc:407] Connection header received
0924 21:14:48.909157 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:48.909164 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:48.909235 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:48.909396 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:48.910051 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:48.910957 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:48.911748 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:48.912310 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:48.912422 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:48.915441 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:48.915444 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:48.915445 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:48.915457 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:48.917467 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:48.918072 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:48.918075 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:48.918075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:48.918185 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:48.918484 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:48.918485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:48.918485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:48.920033 (+ 1548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:48.920199 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:48.920505 (+ 306us) server_negotiation.cc:299] Negotiation successful
0924 21:14:48.920654 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":96,"threads_started":1}
I20260924 21:14:48.938489 20759 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:14:48.939231 20759 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:14:48.945408 20759 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb: Generated new TSK 0
I20260924 21:14:48.945608 20759 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.210.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
I20260924 21:14:48.973544 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.129:0
--local_ip_for_outbound_sockets=127.6.210.129
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:34097
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:43871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:14:49.054363 20773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:14:49.054574 20773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:14:49.054598 20773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:14:49.055760 20773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:14:49.055802 20773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:14:49.055828 20773 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.129
I20260924 21:14:49.057256 20773 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:43871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:34097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.129
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:14:49.057494 20773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:14:49.057757 20773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:14:49.063115 20781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:49.063115 20778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:14:49.063241 20773 server_base.cc:1034] running on GCE node
W20260924 21:14:49.063156 20779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:14:49.063529 20773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:14:49.063792 20773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:14:49.064953 20773 hybrid_clock.cc:648] HybridClock initialized: now 1790284489064930 us; error 35 us; skew 500 ppm
Sep 24 21:14:49 dist-test-slave-q276 krb5kdc[20666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284489, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:14:49.067557 20773 init.cc:377] Logged in from keytab as kudu/127.6.210.129@KRBTEST.COM (short username kudu)
I20260924 21:14:49.068056 20773 webserver.cc:460] Webserver started at http://127.6.210.129:42583/ using document root <none> and password file <none>
I20260924 21:14:49.068307 20773 fs_manager.cc:359] Metadata directory not provided
I20260924 21:14:49.068353 20773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:14:49.068523 20773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:14:49.069563 20773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/data/instance:
uuid: "602408af328941bcbd447c7246078ff0"
format_stamp: "Formatted at 2026-09-24 21:14:49 on dist-test-slave-q276"
server_key: "4bd0f6f357f1beb76fdd68011a669b3d"
server_key_iv: "3094b15333c4ea467092867d3661a6a5"
server_key_version: "encryptionkey@0"
I20260924 21:14:49.069926 20773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance:
uuid: "602408af328941bcbd447c7246078ff0"
format_stamp: "Formatted at 2026-09-24 21:14:49 on dist-test-slave-q276"
server_key: "4bd0f6f357f1beb76fdd68011a669b3d"
server_key_iv: "3094b15333c4ea467092867d3661a6a5"
server_key_version: "encryptionkey@0"
I20260924 21:14:49.071301 20773 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:14:49.071993 20788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:49.072181 20773 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:49.072269 20773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/wal
uuid: "602408af328941bcbd447c7246078ff0"
format_stamp: "Formatted at 2026-09-24 21:14:49 on dist-test-slave-q276"
server_key: "4bd0f6f357f1beb76fdd68011a669b3d"
server_key_iv: "3094b15333c4ea467092867d3661a6a5"
server_key_version: "encryptionkey@0"
I20260924 21:14:49.072340 20773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:14:49.102065 20773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:14:49.104027 20773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:14:49.104204 20773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:14:49.104458 20773 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:14:49.104785 20773 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:14:49.104820 20773 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:49.104841 20773 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:14:49.104890 20773 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:49.110788 20773 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.129:43583
I20260924 21:14:49.110836 20901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.129:43583 every 8 connection(s)
I20260924 21:14:49.111155 20773 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
Sep 24 21:14:49 dist-test-slave-q276 krb5kdc[20666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284489, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:14:49.119009 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 20773
I20260924 21:14:49.119131 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance
I20260924 21:14:49.119305 6986 external_mini_cluster.cc:1428] Setting key 61fadcd97ddb949d45f7422b304cb117
I20260924 21:14:49.120661 20750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:49.111870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:43991 (local address 127.6.210.190:34097)
0924 21:14:49.111959 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:49.111961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:49.112164 (+ 203us) server_negotiation.cc:407] Connection header received
0924 21:14:49.112641 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:49.112642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:49.112651 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:49.112710 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:49.113747 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.114433 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.115482 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.115677 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.115730 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:49.117765 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:49.117766 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:49.117766 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:49.117773 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:49.119072 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:49.119542 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:49.119543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:49.119543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:49.119617 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:49.119955 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:49.119955 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:49.119956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:49.120099 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:49.120197 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:49.120517 (+ 320us) server_negotiation.cc:299] Negotiation successful
0924 21:14:49.120595 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260924 21:14:49.120977 20904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:49.111837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.129:43991)
0924 21:14:49.112036 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:49.112079 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:49.112559 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:49.112779 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:49.112782 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:49.113442 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:14:49.113663 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:49.113669 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.114511 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.114512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:49.115412 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:49.115414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.115748 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.115749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:49.115845 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:49.116286 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:14:49.116294 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:14:49.117677 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:14:49.119165 (+ 1488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:49.119172 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:49.119174 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:49.119461 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:49.119784 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:49.119785 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:49.119785 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:49.119886 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:49.120190 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:14:49.120197 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:14:49.120358 (+ 161us) client_negotiation.cc:769] Sending connection context
0924 21:14:49.120496 (+ 138us) client_negotiation.cc:240] Negotiation successful
0924 21:14:49.120672 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":91,"threads_started":1}
I20260924 21:14:49.121925 20902 heartbeater.cc:347] Connected to a master server at 127.6.210.190:34097
I20260924 21:14:49.122032 20902 heartbeater.cc:464] Registering TS with master...
I20260924 21:14:49.122212 20902 heartbeater.cc:511] Master 127.6.210.190:34097 requested a full tablet report, sending...
I20260924 21:14:49.122834 20710 ts_manager.cc:194] Registered new tserver with Master: 602408af328941bcbd447c7246078ff0 (127.6.210.129:43583)
I20260924 21:14:49.123483 20710 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.129@KRBTEST.COM'} at 127.6.210.129:43991
WARNING: no policy specified for kudu/127.6.210.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
I20260924 21:14:49.144809 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.130:0
--local_ip_for_outbound_sockets=127.6.210.130
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:34097
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:43871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:14:49.224789 20907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:14:49.225039 20907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:14:49.225076 20907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:14:49.226240 20907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:14:49.226332 20907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:14:49.226378 20907 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.130
I20260924 21:14:49.227861 20907 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:43871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:34097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.130
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:14:49.228145 20907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:14:49.228377 20907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:14:49.233485 20915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:49.233525 20913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:49.233632 20912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:14:49.233731 20907 server_base.cc:1034] running on GCE node
I20260924 21:14:49.233898 20907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:14:49.234090 20907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:14:49.235229 20907 hybrid_clock.cc:648] HybridClock initialized: now 1790284489235206 us; error 38 us; skew 500 ppm
Sep 24 21:14:49 dist-test-slave-q276 krb5kdc[20666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284489, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:14:49.237627 20907 init.cc:377] Logged in from keytab as kudu/127.6.210.130@KRBTEST.COM (short username kudu)
I20260924 21:14:49.238073 20907 webserver.cc:460] Webserver started at http://127.6.210.130:46845/ using document root <none> and password file <none>
I20260924 21:14:49.238354 20907 fs_manager.cc:359] Metadata directory not provided
I20260924 21:14:49.238423 20907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:14:49.238580 20907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:14:49.239678 20907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/data/instance:
uuid: "0993dfa60a2e4cef9e4246312d657aba"
format_stamp: "Formatted at 2026-09-24 21:14:49 on dist-test-slave-q276"
server_key: "a62cfb61c93c82765336407613e8ade0"
server_key_iv: "26fa3baf3013f98560b86430e6080281"
server_key_version: "encryptionkey@0"
I20260924 21:14:49.240044 20907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance:
uuid: "0993dfa60a2e4cef9e4246312d657aba"
format_stamp: "Formatted at 2026-09-24 21:14:49 on dist-test-slave-q276"
server_key: "a62cfb61c93c82765336407613e8ade0"
server_key_iv: "26fa3baf3013f98560b86430e6080281"
server_key_version: "encryptionkey@0"
I20260924 21:14:49.241312 20907 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:14:49.241978 20922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:49.242138 20907 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:14:49.242234 20907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/wal
uuid: "0993dfa60a2e4cef9e4246312d657aba"
format_stamp: "Formatted at 2026-09-24 21:14:49 on dist-test-slave-q276"
server_key: "a62cfb61c93c82765336407613e8ade0"
server_key_iv: "26fa3baf3013f98560b86430e6080281"
server_key_version: "encryptionkey@0"
I20260924 21:14:49.242343 20907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:14:49.252887 20907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:14:49.255296 20907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:14:49.255460 20907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:14:49.255690 20907 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:14:49.256009 20907 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:14:49.256068 20907 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:49.256110 20907 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:14:49.256138 20907 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:49.261910 20907 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.130:40125
I20260924 21:14:49.261988 21035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.130:40125 every 8 connection(s)
I20260924 21:14:49.262360 20907 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
Sep 24 21:14:49 dist-test-slave-q276 krb5kdc[20666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284489, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:14:49.269721 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 20907
I20260924 21:14:49.270007 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance
I20260924 21:14:49.270239 6986 external_mini_cluster.cc:1428] Setting key 8c06d14be316a85c791c6a5c39c287ca
I20260924 21:14:49.272778 20750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:49.263158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:36609 (local address 127.6.210.190:34097)
0924 21:14:49.263202 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:49.263205 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:49.263390 (+ 185us) server_negotiation.cc:407] Connection header received
0924 21:14:49.263926 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:49.263928 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:49.263939 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:49.264009 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:49.265017 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.265538 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.266660 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.266862 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.266948 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:49.269289 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:49.269291 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:49.269291 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:49.269301 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:49.271291 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:49.271714 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:49.271715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:49.271716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:49.271786 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:49.272030 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:49.272030 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:49.272031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:49.272194 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:49.272271 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:49.272516 (+ 245us) server_negotiation.cc:299] Negotiation successful
0924 21:14:49.272615 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 21:14:49.272913 21038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:49.263117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.130:36609)
0924 21:14:49.263260 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:49.263306 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:49.263816 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:49.264109 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:49.264119 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:49.264694 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:14:49.264889 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:49.264894 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.265640 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.265641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:49.266559 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:49.266561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.266960 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.266960 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:49.267096 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:49.267647 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:14:49.267655 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:14:49.269163 (+ 1508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:14:49.271397 (+ 2234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:49.271403 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:49.271404 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:49.271636 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:49.271859 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:49.271860 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:49.271860 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:49.271959 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:49.272261 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:14:49.272265 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:14:49.272383 (+ 118us) client_negotiation.cc:769] Sending connection context
0924 21:14:49.272467 (+ 84us) client_negotiation.cc:240] Negotiation successful
0924 21:14:49.272635 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":69,"threads_started":1}
I20260924 21:14:49.273545 21036 heartbeater.cc:347] Connected to a master server at 127.6.210.190:34097
I20260924 21:14:49.273662 21036 heartbeater.cc:464] Registering TS with master...
I20260924 21:14:49.273861 21036 heartbeater.cc:511] Master 127.6.210.190:34097 requested a full tablet report, sending...
I20260924 21:14:49.274322 20710 ts_manager.cc:194] Registered new tserver with Master: 0993dfa60a2e4cef9e4246312d657aba (127.6.210.130:40125)
I20260924 21:14:49.274698 20710 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.130@KRBTEST.COM'} at 127.6.210.130:36609
WARNING: no policy specified for kudu/127.6.210.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
I20260924 21:14:49.296970 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.131:0
--local_ip_for_outbound_sockets=127.6.210.131
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:34097
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:43871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:14:49.378660 21041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:14:49.378867 21041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:14:49.378894 21041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:14:49.380092 21041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:14:49.380136 21041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:14:49.380189 21041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.131
I20260924 21:14:49.381707 21041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:43871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:34097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.131
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:14:49.381942 21041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:14:49.382179 21041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:14:49.387202 21046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:14:49.387302 21041 server_base.cc:1034] running on GCE node
W20260924 21:14:49.387194 21047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:49.387487 21049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:14:49.387715 21041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:14:49.387967 21041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:14:49.389112 21041 hybrid_clock.cc:648] HybridClock initialized: now 1790284489389100 us; error 26 us; skew 500 ppm
Sep 24 21:14:49 dist-test-slave-q276 krb5kdc[20666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284489, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:14:49.391893 21041 init.cc:377] Logged in from keytab as kudu/127.6.210.131@KRBTEST.COM (short username kudu)
I20260924 21:14:49.392385 21041 webserver.cc:460] Webserver started at http://127.6.210.131:45451/ using document root <none> and password file <none>
I20260924 21:14:49.392623 21041 fs_manager.cc:359] Metadata directory not provided
I20260924 21:14:49.392670 21041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:14:49.392834 21041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:14:49.393909 21041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/data/instance:
uuid: "f910de0a3bb54e69af8579664236d51b"
format_stamp: "Formatted at 2026-09-24 21:14:49 on dist-test-slave-q276"
server_key: "31e4d1bc6d022e31058502c8ac4546f9"
server_key_iv: "fb778e48bcd677f0158db473f801126d"
server_key_version: "encryptionkey@0"
I20260924 21:14:49.394312 21041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance:
uuid: "f910de0a3bb54e69af8579664236d51b"
format_stamp: "Formatted at 2026-09-24 21:14:49 on dist-test-slave-q276"
server_key: "31e4d1bc6d022e31058502c8ac4546f9"
server_key_iv: "fb778e48bcd677f0158db473f801126d"
server_key_version: "encryptionkey@0"
I20260924 21:14:49.395632 21041 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:14:49.396349 21056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:49.396520 21041 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:14:49.396618 21041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/wal
uuid: "f910de0a3bb54e69af8579664236d51b"
format_stamp: "Formatted at 2026-09-24 21:14:49 on dist-test-slave-q276"
server_key: "31e4d1bc6d022e31058502c8ac4546f9"
server_key_iv: "fb778e48bcd677f0158db473f801126d"
server_key_version: "encryptionkey@0"
I20260924 21:14:49.396700 21041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:14:49.433581 21041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:14:49.437155 21041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:14:49.437316 21041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:14:49.437613 21041 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:14:49.437973 21041 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:14:49.438021 21041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:49.438074 21041 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:14:49.438097 21041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:49.443908 21041 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.131:46207
I20260924 21:14:49.444002 21169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.131:46207 every 8 connection(s)
I20260924 21:14:49.444377 21041 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
Sep 24 21:14:49 dist-test-slave-q276 krb5kdc[20666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284489, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:14:49.452598 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 21041
I20260924 21:14:49.452687 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance
I20260924 21:14:49.452859 6986 external_mini_cluster.cc:1428] Setting key 1bcefb964728041b2faf28e2866f6cd3
I20260924 21:14:49.453989 20750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:49.445201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:48979 (local address 127.6.210.190:34097)
0924 21:14:49.445250 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:49.445252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:49.445517 (+ 265us) server_negotiation.cc:407] Connection header received
0924 21:14:49.446137 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:49.446138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:49.446149 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:49.446201 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:49.447210 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.447703 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.448661 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.448843 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.448898 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:49.451067 (+ 2169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:49.451069 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:49.451069 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:49.451076 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:49.452520 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:49.452973 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:49.452974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:49.452975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:49.453039 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:49.453325 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:49.453326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:49.453326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:49.453471 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:49.453543 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:49.453831 (+ 288us) server_negotiation.cc:299] Negotiation successful
0924 21:14:49.453918 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 21:14:49.454221 21172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:49.445158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.131:48979)
0924 21:14:49.445345 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:49.445397 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:49.446025 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:49.446310 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:49.446314 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:49.446922 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:14:49.447123 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:49.447129 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.447785 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.447785 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:49.448582 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:49.448584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.448917 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.448918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:49.449020 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:49.449537 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:14:49.449545 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:14:49.450972 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:14:49.452601 (+ 1629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:49.452606 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:49.452607 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:49.452867 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:49.453117 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:49.453118 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:49.453118 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:49.453245 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:49.453551 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:14:49.453556 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:14:49.453675 (+ 119us) client_negotiation.cc:769] Sending connection context
0924 21:14:49.453796 (+ 121us) client_negotiation.cc:240] Negotiation successful
0924 21:14:49.453948 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":95,"threads_started":1}
I20260924 21:14:49.454718 21170 heartbeater.cc:347] Connected to a master server at 127.6.210.190:34097
I20260924 21:14:49.454820 21170 heartbeater.cc:464] Registering TS with master...
I20260924 21:14:49.454994 21170 heartbeater.cc:511] Master 127.6.210.190:34097 requested a full tablet report, sending...
I20260924 21:14:49.455385 20710 ts_manager.cc:194] Registered new tserver with Master: f910de0a3bb54e69af8579664236d51b (127.6.210.131:46207)
I20260924 21:14:49.455746 20710 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.131@KRBTEST.COM'} at 127.6.210.131:48979
I20260924 21:14:49.464896 6986 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:14:49.472632 20750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:49.466238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46078 (local address 127.6.210.190:34097)
0924 21:14:49.466344 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:49.466346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:49.466495 (+ 149us) server_negotiation.cc:407] Connection header received
0924 21:14:49.466545 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:49.466546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:49.466556 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:49.466624 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:49.467212 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.467720 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.468504 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.468696 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.468782 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:49.470089 (+ 1307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:49.470090 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:49.470091 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:49.470099 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:49.471348 (+ 1249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:49.471764 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:49.471765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:49.471766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:49.471827 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:49.472058 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:49.472059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:49.472059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:49.472219 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:49.472330 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:49.472440 (+ 110us) server_negotiation.cc:299] Negotiation successful
0924 21:14:49.472524 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260924 21:14:49.474402 20710 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46078:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:14:49.476235 20710 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:14:49.482939 21181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:49.478617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:40125 (local address 127.0.0.1:36346)
0924 21:14:49.479080 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:49.479088 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:49.479113 (+ 25us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:49.479466 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:49.479470 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:49.479472 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:14:49.479546 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:49.479548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.480889 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.480890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:49.481929 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:49.481931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.482678 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.482679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:49.482742 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:49.482747 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 21:14:49.482805 (+ 58us) client_negotiation.cc:240] Negotiation successful
0924 21:14:49.482851 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"mutex_wait_us":7,"thread_start_us":91,"threads_started":1}
I20260924 21:14:49.482951 21184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:49.478712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36346 (local address 127.6.210.130:40125)
0924 21:14:49.479284 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:49.479287 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:49.479296 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:14:49.479334 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:49.479336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:49.479351 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:49.479628 (+ 277us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:14:49.479702 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.480779 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.482012 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.482579 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.482673 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:49.482726 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:49.482830 (+ 104us) server_negotiation.cc:299] Negotiation successful
0924 21:14:49.482869 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":84,"threads_started":1}
I20260924 21:14:49.483050 21186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:49.479076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60176 (local address 127.6.210.129:43583)
0924 21:14:49.479221 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:49.479226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:49.479237 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:14:49.479279 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:49.479282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:49.479299 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:49.479394 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:14:49.479630 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.480350 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.481682 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.482450 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.482518 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:49.482573 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:49.482900 (+ 327us) server_negotiation.cc:299] Negotiation successful
0924 21:14:49.482941 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":77,"threads_started":1}
I20260924 21:14:49.483163 21182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:49.478617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:43583 (local address 127.0.0.1:60176)
0924 21:14:49.478895 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:49.478939 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:49.478993 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:49.479484 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:49.479484 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:49.479486 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:14:49.479549 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:49.479550 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.480452 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.480453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:49.481595 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:49.481598 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.482608 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.482609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:49.482690 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:49.482703 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 21:14:49.482773 (+ 70us) client_negotiation.cc:240] Negotiation successful
0924 21:14:49.482830 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":5,"thread_start_us":58,"threads_started":1}
I20260924 21:14:49.483667 21185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:49.478841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60550 (local address 127.6.210.131:46207)
0924 21:14:49.478999 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:49.479004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:49.479294 (+ 290us) server_negotiation.cc:407] Connection header received
0924 21:14:49.479343 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:49.479346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:49.479363 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:49.479436 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:14:49.479670 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.480825 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.482042 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.482759 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.482815 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:49.482860 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:49.483546 (+ 686us) server_negotiation.cc:299] Negotiation successful
0924 21:14:49.483590 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
I20260924 21:14:49.483640 20970 tablet_service.cc:1467] Processing CreateTablet for tablet f64b56646fb7424388229f2bf3d109f4 (DEFAULT_TABLE table=test-table [id=1fca48b83b374290a6faefb3ff9a60ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:14:49.483863 21183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:49.478641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:46207 (local address 127.0.0.1:60550)
0924 21:14:49.479159 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:49.479164 (+ 5us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:49.479224 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:49.479521 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:49.479521 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:49.479523 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:14:49.479600 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:49.479602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.480932 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.480932 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:49.481948 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:49.481951 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.482892 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.482893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:49.482946 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:49.482950 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:14:49.483748 (+ 798us) client_negotiation.cc:240] Negotiation successful
0924 21:14:49.483791 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"mutex_wait_us":2,"thread_start_us":70,"threads_started":1}
I20260924 21:14:49.483855 20836 tablet_service.cc:1467] Processing CreateTablet for tablet f64b56646fb7424388229f2bf3d109f4 (DEFAULT_TABLE table=test-table [id=1fca48b83b374290a6faefb3ff9a60ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:14:49.484045 20970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f64b56646fb7424388229f2bf3d109f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:14:49.484136 20836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f64b56646fb7424388229f2bf3d109f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:14:49.484547 21104 tablet_service.cc:1467] Processing CreateTablet for tablet f64b56646fb7424388229f2bf3d109f4 (DEFAULT_TABLE table=test-table [id=1fca48b83b374290a6faefb3ff9a60ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:14:49.484825 21104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f64b56646fb7424388229f2bf3d109f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:14:49.486555 21187 tablet_bootstrap.cc:492] T f64b56646fb7424388229f2bf3d109f4 P 0993dfa60a2e4cef9e4246312d657aba: Bootstrap starting.
I20260924 21:14:49.487119 21187 tablet_bootstrap.cc:654] T f64b56646fb7424388229f2bf3d109f4 P 0993dfa60a2e4cef9e4246312d657aba: Neither blocks nor log segments found. Creating new log.
I20260924 21:14:49.487455 21188 tablet_bootstrap.cc:492] T f64b56646fb7424388229f2bf3d109f4 P 602408af328941bcbd447c7246078ff0: Bootstrap starting.
I20260924 21:14:49.488008 21189 tablet_bootstrap.cc:492] T f64b56646fb7424388229f2bf3d109f4 P f910de0a3bb54e69af8579664236d51b: Bootstrap starting.
I20260924 21:14:49.488107 21187 log.cc:824] T f64b56646fb7424388229f2bf3d109f4 P 0993dfa60a2e4cef9e4246312d657aba: Log is configured to *not* fsync() on all Append() calls
I20260924 21:14:49.488137 21188 tablet_bootstrap.cc:654] T f64b56646fb7424388229f2bf3d109f4 P 602408af328941bcbd447c7246078ff0: Neither blocks nor log segments found. Creating new log.
I20260924 21:14:49.488698 21188 log.cc:824] T f64b56646fb7424388229f2bf3d109f4 P 602408af328941bcbd447c7246078ff0: Log is configured to *not* fsync() on all Append() calls
I20260924 21:14:49.488725 21189 tablet_bootstrap.cc:654] T f64b56646fb7424388229f2bf3d109f4 P f910de0a3bb54e69af8579664236d51b: Neither blocks nor log segments found. Creating new log.
I20260924 21:14:49.488962 21187 tablet_bootstrap.cc:492] T f64b56646fb7424388229f2bf3d109f4 P 0993dfa60a2e4cef9e4246312d657aba: No bootstrap required, opened a new log
I20260924 21:14:49.489045 21187 ts_tablet_manager.cc:1397] T f64b56646fb7424388229f2bf3d109f4 P 0993dfa60a2e4cef9e4246312d657aba: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260924 21:14:49.489311 21189 log.cc:824] T f64b56646fb7424388229f2bf3d109f4 P f910de0a3bb54e69af8579664236d51b: Log is configured to *not* fsync() on all Append() calls
I20260924 21:14:49.489437 21188 tablet_bootstrap.cc:492] T f64b56646fb7424388229f2bf3d109f4 P 602408af328941bcbd447c7246078ff0: No bootstrap required, opened a new log
I20260924 21:14:49.489517 21188 ts_tablet_manager.cc:1397] T f64b56646fb7424388229f2bf3d109f4 P 602408af328941bcbd447c7246078ff0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:14:49.490250 21189 tablet_bootstrap.cc:492] T f64b56646fb7424388229f2bf3d109f4 P f910de0a3bb54e69af8579664236d51b: No bootstrap required, opened a new log
I20260924 21:14:49.490345 21189 ts_tablet_manager.cc:1397] T f64b56646fb7424388229f2bf3d109f4 P f910de0a3bb54e69af8579664236d51b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:14:49.491017 21187 raft_consensus.cc:348] T f64b56646fb7424388229f2bf3d109f4 P 0993dfa60a2e4cef9e4246312d657aba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0993dfa60a2e4cef9e4246312d657aba" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40125 } } peers { permanent_uuid: "602408af328941bcbd447c7246078ff0" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43583 } } peers { permanent_uuid: "f910de0a3bb54e69af8579664236d51b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46207 } }
I20260924 21:14:49.491189 21187 raft_consensus.cc:374] T f64b56646fb7424388229f2bf3d109f4 P 0993dfa60a2e4cef9e4246312d657aba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:14:49.491232 21187 raft_consensus.cc:729] T f64b56646fb7424388229f2bf3d109f4 P 0993dfa60a2e4cef9e4246312d657aba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0993dfa60a2e4cef9e4246312d657aba, State: Initialized, Role: FOLLOWER
I20260924 21:14:49.491374 21187 consensus_queue.cc:260] T f64b56646fb7424388229f2bf3d109f4 P 0993dfa60a2e4cef9e4246312d657aba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0993dfa60a2e4cef9e4246312d657aba" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40125 } } peers { permanent_uuid: "602408af328941bcbd447c7246078ff0" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43583 } } peers { permanent_uuid: "f910de0a3bb54e69af8579664236d51b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46207 } }
I20260924 21:14:49.491658 21187 ts_tablet_manager.cc:1428] T f64b56646fb7424388229f2bf3d109f4 P 0993dfa60a2e4cef9e4246312d657aba: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:14:49.491544 21188 raft_consensus.cc:348] T f64b56646fb7424388229f2bf3d109f4 P 602408af328941bcbd447c7246078ff0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0993dfa60a2e4cef9e4246312d657aba" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40125 } } peers { permanent_uuid: "602408af328941bcbd447c7246078ff0" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43583 } } peers { permanent_uuid: "f910de0a3bb54e69af8579664236d51b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46207 } }
I20260924 21:14:49.491715 21188 raft_consensus.cc:374] T f64b56646fb7424388229f2bf3d109f4 P 602408af328941bcbd447c7246078ff0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:14:49.491760 21188 raft_consensus.cc:729] T f64b56646fb7424388229f2bf3d109f4 P 602408af328941bcbd447c7246078ff0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602408af328941bcbd447c7246078ff0, State: Initialized, Role: FOLLOWER
I20260924 21:14:49.491784 21036 heartbeater.cc:503] Master 127.6.210.190:34097 was elected leader, sending a full tablet report...
I20260924 21:14:49.491871 21188 consensus_queue.cc:260] T f64b56646fb7424388229f2bf3d109f4 P 602408af328941bcbd447c7246078ff0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0993dfa60a2e4cef9e4246312d657aba" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40125 } } peers { permanent_uuid: "602408af328941bcbd447c7246078ff0" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43583 } } peers { permanent_uuid: "f910de0a3bb54e69af8579664236d51b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46207 } }
I20260924 21:14:49.492156 20902 heartbeater.cc:503] Master 127.6.210.190:34097 was elected leader, sending a full tablet report...
I20260924 21:14:49.492157 21188 ts_tablet_manager.cc:1428] T f64b56646fb7424388229f2bf3d109f4 P 602408af328941bcbd447c7246078ff0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:14:49.492347 21189 raft_consensus.cc:348] T f64b56646fb7424388229f2bf3d109f4 P f910de0a3bb54e69af8579664236d51b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0993dfa60a2e4cef9e4246312d657aba" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40125 } } peers { permanent_uuid: "602408af328941bcbd447c7246078ff0" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43583 } } peers { permanent_uuid: "f910de0a3bb54e69af8579664236d51b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46207 } }
I20260924 21:14:49.492518 21189 raft_consensus.cc:374] T f64b56646fb7424388229f2bf3d109f4 P f910de0a3bb54e69af8579664236d51b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:14:49.492555 21189 raft_consensus.cc:729] T f64b56646fb7424388229f2bf3d109f4 P f910de0a3bb54e69af8579664236d51b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f910de0a3bb54e69af8579664236d51b, State: Initialized, Role: FOLLOWER
I20260924 21:14:49.492745 21189 consensus_queue.cc:260] T f64b56646fb7424388229f2bf3d109f4 P f910de0a3bb54e69af8579664236d51b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0993dfa60a2e4cef9e4246312d657aba" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40125 } } peers { permanent_uuid: "602408af328941bcbd447c7246078ff0" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43583 } } peers { permanent_uuid: "f910de0a3bb54e69af8579664236d51b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46207 } }
I20260924 21:14:49.493049 21189 ts_tablet_manager.cc:1428] T f64b56646fb7424388229f2bf3d109f4 P f910de0a3bb54e69af8579664236d51b: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260924 21:14:49.493091 21170 heartbeater.cc:503] Master 127.6.210.190:34097 was elected leader, sending a full tablet report...
W20260924 21:14:49.513006 21037 tablet.cc:2369] T f64b56646fb7424388229f2bf3d109f4 P 0993dfa60a2e4cef9e4246312d657aba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:14:49.555465 21195 raft_consensus.cc:482] T f64b56646fb7424388229f2bf3d109f4 P f910de0a3bb54e69af8579664236d51b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:14:49.555683 21195 raft_consensus.cc:504] T f64b56646fb7424388229f2bf3d109f4 P f910de0a3bb54e69af8579664236d51b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0993dfa60a2e4cef9e4246312d657aba" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40125 } } peers { permanent_uuid: "602408af328941bcbd447c7246078ff0" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43583 } } peers { permanent_uuid: "f910de0a3bb54e69af8579664236d51b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46207 } }
I20260924 21:14:49.556110 21195 leader_election.cc:288] T f64b56646fb7424388229f2bf3d109f4 P f910de0a3bb54e69af8579664236d51b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0993dfa60a2e4cef9e4246312d657aba (127.6.210.130:40125), 602408af328941bcbd447c7246078ff0 (127.6.210.129:43583)
I20260924 21:14:49.560333 21186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:49.556388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:49259 (local address 127.6.210.129:43583)
0924 21:14:49.556431 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:49.556433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:49.556667 (+ 234us) server_negotiation.cc:407] Connection header received
0924 21:14:49.556699 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:49.556700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:49.556708 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:49.556759 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:14:49.556944 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.557640 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.559291 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.559880 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.559962 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:49.559998 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:49.560218 (+ 220us) server_negotiation.cc:299] Negotiation successful
0924 21:14:49.560261 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 21:14:49.560333 21184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:49.556424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:42159 (local address 127.6.210.130:40125)
0924 21:14:49.556497 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:49.556499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:49.556508 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:14:49.556543 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:49.556543 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:49.556551 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:49.556598 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:14:49.556838 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.557612 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.559286 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.559880 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.559961 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:49.559970 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:49.560219 (+ 249us) server_negotiation.cc:299] Negotiation successful
0924 21:14:49.560261 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260924 21:14:49.560420 21172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:49.556286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:40125 (local address 127.6.210.131:42159)
0924 21:14:49.556357 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:49.556379 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:49.556440 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:49.556637 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:49.556638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:49.556639 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:14:49.556727 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:49.556731 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.557732 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.557733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:49.559175 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:49.559179 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.560023 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.560025 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:49.560124 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:49.560126 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 21:14:49.560205 (+ 79us) client_negotiation.cc:240] Negotiation successful
0924 21:14:49.560257 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":43}
I20260924 21:14:49.560655 21196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:49.556286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:43583 (local address 127.6.210.131:49259)
0924 21:14:49.556546 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:49.556555 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:49.556630 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:49.556787 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:49.556788 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:49.556790 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:14:49.556854 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:49.556856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.557732 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.557733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:49.559175 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:49.559179 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.560024 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.560030 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:49.560121 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:49.560126 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 21:14:49.560205 (+ 79us) client_negotiation.cc:240] Negotiation successful
0924 21:14:49.560257 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":96,"thread_start_us":105,"threads_started":1}
I20260924 21:14:49.560819 20990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64b56646fb7424388229f2bf3d109f4" candidate_uuid: "f910de0a3bb54e69af8579664236d51b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0993dfa60a2e4cef9e4246312d657aba" is_pre_election: true
I20260924 21:14:49.560999 20990 raft_consensus.cc:2454] T f64b56646fb7424388229f2bf3d109f4 P 0993dfa60a2e4cef9e4246312d657aba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f910de0a3bb54e69af8579664236d51b in term 0.
I20260924 21:14:49.561057 20856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64b56646fb7424388229f2bf3d109f4" candidate_uuid: "f910de0a3bb54e69af8579664236d51b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602408af328941bcbd447c7246078ff0" is_pre_election: true
I20260924 21:14:49.561242 20856 raft_consensus.cc:2454] T f64b56646fb7424388229f2bf3d109f4 P 602408af328941bcbd447c7246078ff0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f910de0a3bb54e69af8579664236d51b in term 0.
I20260924 21:14:49.561249 21059 leader_election.cc:302] T f64b56646fb7424388229f2bf3d109f4 P f910de0a3bb54e69af8579664236d51b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0993dfa60a2e4cef9e4246312d657aba, f910de0a3bb54e69af8579664236d51b; no voters:
I20260924 21:14:49.561379 21195 raft_consensus.cc:2781] T f64b56646fb7424388229f2bf3d109f4 P f910de0a3bb54e69af8579664236d51b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:14:49.561465 21195 raft_consensus.cc:482] T f64b56646fb7424388229f2bf3d109f4 P f910de0a3bb54e69af8579664236d51b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:14:49.561499 21195 raft_consensus.cc:3037] T f64b56646fb7424388229f2bf3d109f4 P f910de0a3bb54e69af8579664236d51b [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:14:49.562409 21195 raft_consensus.cc:504] T f64b56646fb7424388229f2bf3d109f4 P f910de0a3bb54e69af8579664236d51b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0993dfa60a2e4cef9e4246312d657aba" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40125 } } peers { permanent_uuid: "602408af328941bcbd447c7246078ff0" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43583 } } peers { permanent_uuid: "f910de0a3bb54e69af8579664236d51b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46207 } }
I20260924 21:14:49.562579 21195 leader_election.cc:288] T f64b56646fb7424388229f2bf3d109f4 P f910de0a3bb54e69af8579664236d51b [CANDIDATE]: Term 1 election: Requested vote from peers 0993dfa60a2e4cef9e4246312d657aba (127.6.210.130:40125), 602408af328941bcbd447c7246078ff0 (127.6.210.129:43583)
I20260924 21:14:49.562762 20856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64b56646fb7424388229f2bf3d109f4" candidate_uuid: "f910de0a3bb54e69af8579664236d51b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602408af328941bcbd447c7246078ff0"
I20260924 21:14:49.562768 20990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f64b56646fb7424388229f2bf3d109f4" candidate_uuid: "f910de0a3bb54e69af8579664236d51b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0993dfa60a2e4cef9e4246312d657aba"
I20260924 21:14:49.562860 20856 raft_consensus.cc:3037] T f64b56646fb7424388229f2bf3d109f4 P 602408af328941bcbd447c7246078ff0 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:14:49.562860 20990 raft_consensus.cc:3037] T f64b56646fb7424388229f2bf3d109f4 P 0993dfa60a2e4cef9e4246312d657aba [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:14:49.563813 20856 raft_consensus.cc:2454] T f64b56646fb7424388229f2bf3d109f4 P 602408af328941bcbd447c7246078ff0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f910de0a3bb54e69af8579664236d51b in term 1.
I20260924 21:14:49.563822 20990 raft_consensus.cc:2454] T f64b56646fb7424388229f2bf3d109f4 P 0993dfa60a2e4cef9e4246312d657aba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f910de0a3bb54e69af8579664236d51b in term 1.
I20260924 21:14:49.564016 21057 leader_election.cc:302] T f64b56646fb7424388229f2bf3d109f4 P f910de0a3bb54e69af8579664236d51b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602408af328941bcbd447c7246078ff0, f910de0a3bb54e69af8579664236d51b; no voters:
I20260924 21:14:49.564157 21195 raft_consensus.cc:2781] T f64b56646fb7424388229f2bf3d109f4 P f910de0a3bb54e69af8579664236d51b [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:14:49.564329 21195 raft_consensus.cc:686] T f64b56646fb7424388229f2bf3d109f4 P f910de0a3bb54e69af8579664236d51b [term 1 LEADER]: Becoming Leader. State: Replica: f910de0a3bb54e69af8579664236d51b, State: Running, Role: LEADER
I20260924 21:14:49.564455 21195 consensus_queue.cc:237] T f64b56646fb7424388229f2bf3d109f4 P f910de0a3bb54e69af8579664236d51b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0993dfa60a2e4cef9e4246312d657aba" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40125 } } peers { permanent_uuid: "602408af328941bcbd447c7246078ff0" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43583 } } peers { permanent_uuid: "f910de0a3bb54e69af8579664236d51b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46207 } }
I20260924 21:14:49.565337 20711 catalog_manager.cc:5495] T f64b56646fb7424388229f2bf3d109f4 P f910de0a3bb54e69af8579664236d51b reported cstate change: term changed from 0 to 1, leader changed from <none> to f910de0a3bb54e69af8579664236d51b (127.6.210.131). New cstate: current_term: 1 leader_uuid: "f910de0a3bb54e69af8579664236d51b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0993dfa60a2e4cef9e4246312d657aba" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "602408af328941bcbd447c7246078ff0" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f910de0a3bb54e69af8579664236d51b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46207 } health_report { overall_health: HEALTHY } } }
I20260924 21:14:49.578629 21185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:49.576300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60562 (local address 127.6.210.131:46207)
0924 21:14:49.576344 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:49.576346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:49.576355 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:14:49.576400 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:49.576400 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:49.576417 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:49.576474 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:14:49.576570 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.577026 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.577778 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.578007 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.578067 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:49.578177 (+ 110us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:14:49.578425 (+ 248us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:14:49.578497 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:49.578544 (+ 47us) server_negotiation.cc:299] Negotiation successful
0924 21:14:49.578563 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 21:14:49.580410 20990 raft_consensus.cc:1261] T f64b56646fb7424388229f2bf3d109f4 P 0993dfa60a2e4cef9e4246312d657aba [term 1 FOLLOWER]: Refusing update from remote peer f910de0a3bb54e69af8579664236d51b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:14:49.580410 20856 raft_consensus.cc:1261] T f64b56646fb7424388229f2bf3d109f4 P 602408af328941bcbd447c7246078ff0 [term 1 FOLLOWER]: Refusing update from remote peer f910de0a3bb54e69af8579664236d51b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:14:49.580757 21197 consensus_queue.cc:1035] T f64b56646fb7424388229f2bf3d109f4 P f910de0a3bb54e69af8579664236d51b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0993dfa60a2e4cef9e4246312d657aba" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:14:49.580916 21195 consensus_queue.cc:1035] T f64b56646fb7424388229f2bf3d109f4 P f910de0a3bb54e69af8579664236d51b [LEADER]: Connected to new peer: Peer: permanent_uuid: "602408af328941bcbd447c7246078ff0" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:14:49.583405 21202 mvcc.cc:205] Tried to move back new op lower bound from 7333005269318692864 to 7333005269256835072. Current Snapshot: MvccSnapshot[applied={T|T < 7333005269318692864}]
I20260924 21:14:49.583420 21200 mvcc.cc:205] Tried to move back new op lower bound from 7333005269318692864 to 7333005269256835072. Current Snapshot: MvccSnapshot[applied={T|T < 7333005269318692864}]
I20260924 21:14:49.583925 21201 mvcc.cc:205] Tried to move back new op lower bound from 7333005269318692864 to 7333005269256835072. Current Snapshot: MvccSnapshot[applied={T|T < 7333005269318692864}]
I20260924 21:14:49.586418 20710 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:14:49.586532 20710 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:14:49.587260 20710 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb: Sending DeleteTablet for 3 replicas of tablet f64b56646fb7424388229f2bf3d109f4
I20260924 21:14:49.587610 20970 tablet_service.cc:1514] Processing DeleteTablet for tablet f64b56646fb7424388229f2bf3d109f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:14:49 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:36346
I20260924 21:14:49.587669 20836 tablet_service.cc:1514] Processing DeleteTablet for tablet f64b56646fb7424388229f2bf3d109f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:14:49 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:60176
I20260924 21:14:49.587724 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 20773
I20260924 21:14:49.587817 21104 tablet_service.cc:1514] Processing DeleteTablet for tablet f64b56646fb7424388229f2bf3d109f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:14:49 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:60550
I20260924 21:14:49.588369 21211 tablet_replica.cc:330] T f64b56646fb7424388229f2bf3d109f4 P f910de0a3bb54e69af8579664236d51b: stopping tablet replica
I20260924 21:14:49.588480 21211 raft_consensus.cc:2229] T f64b56646fb7424388229f2bf3d109f4 P f910de0a3bb54e69af8579664236d51b [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:14:49.588605 21211 raft_consensus.cc:2258] T f64b56646fb7424388229f2bf3d109f4 P f910de0a3bb54e69af8579664236d51b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:14:49.589038 21211 ts_tablet_manager.cc:1905] T f64b56646fb7424388229f2bf3d109f4 P f910de0a3bb54e69af8579664236d51b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:14:49.591046 21211 ts_tablet_manager.cc:1918] T f64b56646fb7424388229f2bf3d109f4 P f910de0a3bb54e69af8579664236d51b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:14:49.591107 21211 log.cc:1196] T f64b56646fb7424388229f2bf3d109f4 P f910de0a3bb54e69af8579664236d51b: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/wal/wals/f64b56646fb7424388229f2bf3d109f4
I20260924 21:14:49.591380 21211 ts_tablet_manager.cc:1939] T f64b56646fb7424388229f2bf3d109f4 P f910de0a3bb54e69af8579664236d51b: Deleting consensus metadata
I20260924 21:14:49.591778 20696 catalog_manager.cc:4841] TS f910de0a3bb54e69af8579664236d51b (127.6.210.131:46207): tablet f64b56646fb7424388229f2bf3d109f4 (table test-table [id=1fca48b83b374290a6faefb3ff9a60ec]) successfully deleted
I20260924 21:14:49.594406 21210 tablet_replica.cc:330] T f64b56646fb7424388229f2bf3d109f4 P 0993dfa60a2e4cef9e4246312d657aba: stopping tablet replica
I20260924 21:14:49.594550 21210 raft_consensus.cc:2229] T f64b56646fb7424388229f2bf3d109f4 P 0993dfa60a2e4cef9e4246312d657aba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:14:49.594671 21210 raft_consensus.cc:2258] T f64b56646fb7424388229f2bf3d109f4 P 0993dfa60a2e4cef9e4246312d657aba [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 21:14:49.594878 20695 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.210.129:43583 (error 108)
I20260924 21:14:49.595135 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 20907
I20260924 21:14:49.595796 21183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:49.595277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:43583 (local address 127.0.0.1:60186)
0924 21:14:49.595713 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:49.595756 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.129:43583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411}
W20260924 21:14:49.597745 20695 catalog_manager.cc:4584] TS 602408af328941bcbd447c7246078ff0 (127.6.210.129:43583): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f64b56646fb7424388229f2bf3d109f4: Network error: Client connection negotiation failed: client connection to 127.6.210.129:43583: connect: Connection refused (error 111)
I20260924 21:14:49.602079 21183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:49.601915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:40125 (local address 127.0.0.1:36356)
0924 21:14:49.601967 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:49.602041 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.130:40125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260924 21:14:49.602097 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 21041
I20260924 21:14:49.606091 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 20679
I20260924 21:14:49.611219 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:34097
--webserver_interface=127.6.210.190
--webserver_port=44911
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:43871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:34097
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:14:49.691927 21213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:14:49.692145 21213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:14:49.692167 21213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:14:49.693316 21213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:14:49.693360 21213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:14:49.693374 21213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:14:49.693387 21213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:14:49.693399 21213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:14:49.694887 21213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:43871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:34097
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:34097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=44911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:14:49.695142 21213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:14:49.695389 21213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 21:14:49.703718 21213 server_base.cc:1034] running on GCE node
W20260924 21:14:49.703697 21220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:49.703697 21222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:49.703722 21219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:14:49.704231 21213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:14:49.704566 21213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:14:49.705708 21213 hybrid_clock.cc:648] HybridClock initialized: now 1790284489705693 us; error 29 us; skew 500 ppm
Sep 24 21:14:49 dist-test-slave-q276 krb5kdc[20666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284489, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:14:49.713724 21213 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:14:49.714206 21213 webserver.cc:460] Webserver started at http://127.6.210.190:44911/ using document root <none> and password file <none>
I20260924 21:14:49.714480 21213 fs_manager.cc:359] Metadata directory not provided
I20260924 21:14:49.714565 21213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:14:49.715968 21213 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:14:49.716647 21229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:49.716928 21213 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:14:49.717053 21213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "e5faabaa1eee4d84a3504cde1f09eedb"
format_stamp: "Formatted at 2026-09-24 21:14:48 on dist-test-slave-q276"
server_key: "20b16c68c5ce2f11c4de44030da35020"
server_key_iv: "278bc3738ccdbfd86105d88b71a77ccd"
server_key_version: "encryptionkey@0"
I20260924 21:14:49.717469 21213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:14:49.729298 21213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:14:49.731243 21213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:14:49.731405 21213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:14:49.735512 21213 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:34097
I20260924 21:14:49.735544 21281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:34097 every 8 connection(s)
I20260924 21:14:49.735927 21213 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:14:49.737488 21282 sys_catalog.cc:263] Verifying existing consensus state
I20260924 21:14:49.738160 21282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb: Bootstrap starting.
I20260924 21:14:49.746212 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 21213
I20260924 21:14:49.746423 21282 log.cc:824] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb: Log is configured to *not* fsync() on all Append() calls
I20260924 21:14:49.746546 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.129:43583
--local_ip_for_outbound_sockets=127.6.210.129
--tserver_master_addrs=127.6.210.190:34097
--webserver_port=42583
--webserver_interface=127.6.210.129
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:43871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:14:49.748706 21282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 21:14:49.749010 21282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb: Bootstrap complete.
I20260924 21:14:49.751350 21282 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5faabaa1eee4d84a3504cde1f09eedb" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34097 } }
I20260924 21:14:49.751600 21282 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5faabaa1eee4d84a3504cde1f09eedb, State: Initialized, Role: FOLLOWER
I20260924 21:14:49.751778 21282 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5faabaa1eee4d84a3504cde1f09eedb" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34097 } }
I20260924 21:14:49.751854 21282 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:14:49.751911 21282 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:14:49.751973 21282 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [term 1 FOLLOWER]: Advancing to term 2
I20260924 21:14:49.752967 21282 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5faabaa1eee4d84a3504cde1f09eedb" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34097 } }
I20260924 21:14:49.753088 21282 leader_election.cc:302] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5faabaa1eee4d84a3504cde1f09eedb; no voters:
I20260924 21:14:49.753278 21282 leader_election.cc:288] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 21:14:49.753314 21286 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [term 2 FOLLOWER]: Leader election won for term 2
I20260924 21:14:49.753418 21286 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [term 2 LEADER]: Becoming Leader. State: Replica: e5faabaa1eee4d84a3504cde1f09eedb, State: Running, Role: LEADER
I20260924 21:14:49.753522 21286 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5faabaa1eee4d84a3504cde1f09eedb" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34097 } }
I20260924 21:14:49.753561 21282 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:14:49.753793 21288 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5faabaa1eee4d84a3504cde1f09eedb. Latest consensus state: current_term: 2 leader_uuid: "e5faabaa1eee4d84a3504cde1f09eedb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5faabaa1eee4d84a3504cde1f09eedb" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34097 } } }
I20260924 21:14:49.754060 21288 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [sys.catalog]: This master's current role is: LEADER
I20260924 21:14:49.753971 21287 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e5faabaa1eee4d84a3504cde1f09eedb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5faabaa1eee4d84a3504cde1f09eedb" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34097 } } }
I20260924 21:14:49.754356 21287 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [sys.catalog]: This master's current role is: LEADER
I20260924 21:14:49.754426 21296 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:14:49.755199 21296 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:14:49.755311 21296 catalog_manager.cc:1259] Loaded cluster ID: cf8ab8b0807d453abf05c7b382fd51e9
I20260924 21:14:49.755355 21296 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:14:49.755816 21296 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260924 21:14:49.758605 21296 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260924 21:14:49.767057 21296 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:14:49.767292 21296 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb: Loaded TSK: 0
I20260924 21:14:49.767521 21296 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 21:14:49.848491 21285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:14:49.848691 21285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:14:49.848714 21285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:14:49.849901 21285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:14:49.849944 21285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:14:49.849972 21285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.129
I20260924 21:14:49.851531 21285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:43871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.129:43583
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.210.129
--webserver_port=42583
--tserver_master_addrs=127.6.210.190:34097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.129
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:14:49.851765 21285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:14:49.852010 21285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:14:49.857431 21309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:49.857486 21312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:14:49.857596 21285 server_base.cc:1034] running on GCE node
W20260924 21:14:49.857452 21310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:14:49.858078 21285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:14:49.858407 21285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:14:49.859562 21285 hybrid_clock.cc:648] HybridClock initialized: now 1790284489859543 us; error 34 us; skew 500 ppm
Sep 24 21:14:49 dist-test-slave-q276 krb5kdc[20666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284489, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:14:49.862121 21285 init.cc:377] Logged in from keytab as kudu/127.6.210.129@KRBTEST.COM (short username kudu)
I20260924 21:14:49.862711 21285 webserver.cc:460] Webserver started at http://127.6.210.129:42583/ using document root <none> and password file <none>
I20260924 21:14:49.862949 21285 fs_manager.cc:359] Metadata directory not provided
I20260924 21:14:49.863032 21285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:14:49.864408 21285 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:14:49.865177 21319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:49.865406 21285 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:14:49.865507 21285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/wal
uuid: "602408af328941bcbd447c7246078ff0"
format_stamp: "Formatted at 2026-09-24 21:14:49 on dist-test-slave-q276"
server_key: "4bd0f6f357f1beb76fdd68011a669b3d"
server_key_iv: "3094b15333c4ea467092867d3661a6a5"
server_key_version: "encryptionkey@0"
I20260924 21:14:49.865967 21285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:14:49.883834 21285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:14:49.886363 21285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:14:49.886526 21285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:14:49.886794 21285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:14:49.887280 21326 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 21:14:49.888731 21285 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 21:14:49.888792 21285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260924 21:14:49.888829 21285 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 21:14:49.889683 21285 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 21:14:49.889730 21285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260924 21:14:49.889807 21326 tablet_bootstrap.cc:492] T f64b56646fb7424388229f2bf3d109f4 P 602408af328941bcbd447c7246078ff0: Bootstrap starting.
I20260924 21:14:49.896687 21285 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.129:43583
I20260924 21:14:49.896888 21433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.129:43583 every 8 connection(s)
I20260924 21:14:49.897132 21285 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
I20260924 21:14:49.900674 21326 log.cc:824] T f64b56646fb7424388229f2bf3d109f4 P 602408af328941bcbd447c7246078ff0: Log is configured to *not* fsync() on all Append() calls
I20260924 21:14:49.902151 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 21285
I20260924 21:14:49.902451 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.130:40125
--local_ip_for_outbound_sockets=127.6.210.130
--tserver_master_addrs=127.6.210.190:34097
--webserver_port=46845
--webserver_interface=127.6.210.130
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:43871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:14:49.903491 21326 tablet_bootstrap.cc:492] T f64b56646fb7424388229f2bf3d109f4 P 602408af328941bcbd447c7246078ff0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 21:14:49.903802 21326 tablet_bootstrap.cc:492] T f64b56646fb7424388229f2bf3d109f4 P 602408af328941bcbd447c7246078ff0: Bootstrap complete.
I20260924 21:14:49.905018 21326 ts_tablet_manager.cc:1397] T f64b56646fb7424388229f2bf3d109f4 P 602408af328941bcbd447c7246078ff0: Time spent bootstrapping tablet: real 0.015s user 0.011s sys 0.004s
I20260924 21:14:49.906477 21326 raft_consensus.cc:348] T f64b56646fb7424388229f2bf3d109f4 P 602408af328941bcbd447c7246078ff0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0993dfa60a2e4cef9e4246312d657aba" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40125 } } peers { permanent_uuid: "602408af328941bcbd447c7246078ff0" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43583 } } peers { permanent_uuid: "f910de0a3bb54e69af8579664236d51b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46207 } }
I20260924 21:14:49.906760 21326 raft_consensus.cc:729] T f64b56646fb7424388229f2bf3d109f4 P 602408af328941bcbd447c7246078ff0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602408af328941bcbd447c7246078ff0, State: Initialized, Role: FOLLOWER
Sep 24 21:14:49 dist-test-slave-q276 krb5kdc[20666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284489, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:14:49.910415 21326 consensus_queue.cc:260] T f64b56646fb7424388229f2bf3d109f4 P 602408af328941bcbd447c7246078ff0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0993dfa60a2e4cef9e4246312d657aba" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40125 } } peers { permanent_uuid: "602408af328941bcbd447c7246078ff0" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43583 } } peers { permanent_uuid: "f910de0a3bb54e69af8579664236d51b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46207 } }
I20260924 21:14:49.910864 21326 ts_tablet_manager.cc:1428] T f64b56646fb7424388229f2bf3d109f4 P 602408af328941bcbd447c7246078ff0: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260924 21:14:49.913542 21436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:49.897981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.129:46913)
0924 21:14:49.898376 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:49.898425 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:49.899211 (+ 786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:49.899866 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:49.899871 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:49.900261 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:14:49.900540 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:49.900546 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.904129 (+ 3583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.904131 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:49.905462 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:49.905465 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.906324 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.906325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:49.906471 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:49.907114 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:14:49.907125 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:14:49.909004 (+ 1879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:14:49.911648 (+ 2644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:49.911654 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:49.911657 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:49.911956 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:49.912281 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:49.912282 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:49.912282 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:49.912418 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:49.912915 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:14:49.912921 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:14:49.913069 (+ 148us) client_negotiation.cc:769] Sending connection context
0924 21:14:49.913175 (+ 106us) client_negotiation.cc:240] Negotiation successful
0924 21:14:49.913343 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":100,"threads_started":1}
I20260924 21:14:49.913650 21437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:49.898103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:46913 (local address 127.6.210.190:34097)
0924 21:14:49.898338 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:49.898348 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:49.898559 (+ 211us) server_negotiation.cc:407] Connection header received
0924 21:14:49.899501 (+ 942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:49.899509 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:49.899569 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:49.899717 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:49.900678 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.903996 (+ 3318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.905583 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.906182 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.906328 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:49.909132 (+ 2804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:49.909137 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:49.909137 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:49.909151 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:49.911551 (+ 2400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:49.912057 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:49.912061 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:49.912061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:49.912193 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:49.912497 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:49.912498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:49.912498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:49.912835 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:49.912932 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:49.913262 (+ 330us) server_negotiation.cc:299] Negotiation successful
0924 21:14:49.913414 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":104,"threads_started":1}
I20260924 21:14:49.914635 21434 heartbeater.cc:347] Connected to a master server at 127.6.210.190:34097
I20260924 21:14:49.914744 21434 heartbeater.cc:464] Registering TS with master...
I20260924 21:14:49.914942 21434 heartbeater.cc:511] Master 127.6.210.190:34097 requested a full tablet report, sending...
I20260924 21:14:49.915766 21246 ts_manager.cc:194] Registered new tserver with Master: 602408af328941bcbd447c7246078ff0 (127.6.210.129:43583)
I20260924 21:14:49.917008 21246 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.129@KRBTEST.COM'} at 127.6.210.129:46913
I20260924 21:14:49.918327 21434 heartbeater.cc:503] Master 127.6.210.190:34097 was elected leader, sending a full tablet report...
I20260924 21:14:49.922334 21442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:49.917502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60196 (local address 127.6.210.129:43583)
0924 21:14:49.918818 (+ 1316us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:49.918822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:49.918831 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:14:49.918861 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:49.918863 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:49.918878 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:49.918956 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:14:49.919128 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.920231 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.921225 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.921915 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.921998 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:49.922051 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:49.922189 (+ 138us) server_negotiation.cc:299] Negotiation successful
0924 21:14:49.922227 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1289,"thread_start_us":96,"threads_started":1}
I20260924 21:14:49.922329 21441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:49.917416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:43583 (local address 127.0.0.1:60196)
0924 21:14:49.917609 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:49.917645 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:49.917707 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:49.918981 (+ 1274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:49.918984 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:49.918986 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:14:49.919049 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:49.919050 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.920345 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.920346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:49.921149 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:49.921151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:49.922011 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:49.922012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:49.922085 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:49.922095 (+ 10us) client_negotiation.cc:769] Sending connection context
0924 21:14:49.922164 (+ 69us) client_negotiation.cc:240] Negotiation successful
0924 21:14:49.922205 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":103,"threads_started":1}
I20260924 21:14:49.922787 21363 tablet_service.cc:1514] Processing DeleteTablet for tablet f64b56646fb7424388229f2bf3d109f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:14:49 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:60196
I20260924 21:14:49.923398 21443 tablet_replica.cc:330] stopping tablet replica
I20260924 21:14:49.923521 21443 raft_consensus.cc:2229] T f64b56646fb7424388229f2bf3d109f4 P 602408af328941bcbd447c7246078ff0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:14:49.923607 21443 raft_consensus.cc:2258] T f64b56646fb7424388229f2bf3d109f4 P 602408af328941bcbd447c7246078ff0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:14:49.923900 21443 ts_tablet_manager.cc:1905] T f64b56646fb7424388229f2bf3d109f4 P 602408af328941bcbd447c7246078ff0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:14:49.925822 21443 ts_tablet_manager.cc:1918] T f64b56646fb7424388229f2bf3d109f4 P 602408af328941bcbd447c7246078ff0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:14:49.925913 21443 log.cc:1196] T f64b56646fb7424388229f2bf3d109f4 P 602408af328941bcbd447c7246078ff0: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/wal/wals/f64b56646fb7424388229f2bf3d109f4
I20260924 21:14:49.926151 21443 ts_tablet_manager.cc:1939] T f64b56646fb7424388229f2bf3d109f4 P 602408af328941bcbd447c7246078ff0: Deleting consensus metadata
I20260924 21:14:49.926705 21230 catalog_manager.cc:4841] TS 602408af328941bcbd447c7246078ff0 (127.6.210.129:43583): tablet f64b56646fb7424388229f2bf3d109f4 (table test-table [id=1fca48b83b374290a6faefb3ff9a60ec]) successfully deleted
W20260924 21:14:49.991706 21439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:14:49.991909 21439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:14:49.991931 21439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:14:49.992992 21439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:14:49.993026 21439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:14:49.993052 21439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.130
I20260924 21:14:49.994443 21439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:43871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.130:40125
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.210.130
--webserver_port=46845
--tserver_master_addrs=127.6.210.190:34097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.130
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:14:49.994625 21439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:14:49.994796 21439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:14:49.999626 21451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:49.999614 21448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:49.999629 21449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:14:49.999828 21439 server_base.cc:1034] running on GCE node
I20260924 21:14:50.000039 21439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:14:50.000264 21439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:14:50.001417 21439 hybrid_clock.cc:648] HybridClock initialized: now 1790284490001404 us; error 28 us; skew 500 ppm
Sep 24 21:14:50 dist-test-slave-q276 krb5kdc[20666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284490, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:14:50.003911 21439 init.cc:377] Logged in from keytab as kudu/127.6.210.130@KRBTEST.COM (short username kudu)
I20260924 21:14:50.004351 21439 webserver.cc:460] Webserver started at http://127.6.210.130:46845/ using document root <none> and password file <none>
I20260924 21:14:50.004585 21439 fs_manager.cc:359] Metadata directory not provided
I20260924 21:14:50.004668 21439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:14:50.006013 21439 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:14:50.006708 21458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:50.006868 21439 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:50.006965 21439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/wal
uuid: "0993dfa60a2e4cef9e4246312d657aba"
format_stamp: "Formatted at 2026-09-24 21:14:49 on dist-test-slave-q276"
server_key: "a62cfb61c93c82765336407613e8ade0"
server_key_iv: "26fa3baf3013f98560b86430e6080281"
server_key_version: "encryptionkey@0"
I20260924 21:14:50.007323 21439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:14:50.020119 21439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:14:50.022131 21439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:14:50.022351 21439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:14:50.022596 21439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:14:50.023068 21465 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 21:14:50.024683 21439 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 21:14:50.024746 21439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260924 21:14:50.024792 21439 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 21:14:50.025422 21439 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 21:14:50.025481 21439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260924 21:14:50.025544 21465 tablet_bootstrap.cc:492] T f64b56646fb7424388229f2bf3d109f4 P 0993dfa60a2e4cef9e4246312d657aba: Bootstrap starting.
I20260924 21:14:50.032732 21439 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.130:40125
I20260924 21:14:50.032847 21572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.130:40125 every 8 connection(s)
I20260924 21:14:50.033209 21439 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
I20260924 21:14:50.034749 21465 log.cc:824] T f64b56646fb7424388229f2bf3d109f4 P 0993dfa60a2e4cef9e4246312d657aba: Log is configured to *not* fsync() on all Append() calls
I20260924 21:14:50.036752 21465 tablet_bootstrap.cc:492] T f64b56646fb7424388229f2bf3d109f4 P 0993dfa60a2e4cef9e4246312d657aba: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 21:14:50.037030 21465 tablet_bootstrap.cc:492] T f64b56646fb7424388229f2bf3d109f4 P 0993dfa60a2e4cef9e4246312d657aba: Bootstrap complete.
I20260924 21:14:50.037930 21465 ts_tablet_manager.cc:1397] T f64b56646fb7424388229f2bf3d109f4 P 0993dfa60a2e4cef9e4246312d657aba: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.007s
I20260924 21:14:50.039080 21465 raft_consensus.cc:348] T f64b56646fb7424388229f2bf3d109f4 P 0993dfa60a2e4cef9e4246312d657aba [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0993dfa60a2e4cef9e4246312d657aba" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40125 } } peers { permanent_uuid: "602408af328941bcbd447c7246078ff0" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43583 } } peers { permanent_uuid: "f910de0a3bb54e69af8579664236d51b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46207 } }
I20260924 21:14:50.039426 21465 raft_consensus.cc:729] T f64b56646fb7424388229f2bf3d109f4 P 0993dfa60a2e4cef9e4246312d657aba [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0993dfa60a2e4cef9e4246312d657aba, State: Initialized, Role: FOLLOWER
I20260924 21:14:50.039582 21465 consensus_queue.cc:260] T f64b56646fb7424388229f2bf3d109f4 P 0993dfa60a2e4cef9e4246312d657aba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0993dfa60a2e4cef9e4246312d657aba" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40125 } } peers { permanent_uuid: "602408af328941bcbd447c7246078ff0" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43583 } } peers { permanent_uuid: "f910de0a3bb54e69af8579664236d51b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46207 } }
Sep 24 21:14:50 dist-test-slave-q276 krb5kdc[20666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284490, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:14:50.039851 21465 ts_tablet_manager.cc:1428] T f64b56646fb7424388229f2bf3d109f4 P 0993dfa60a2e4cef9e4246312d657aba: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20260924 21:14:50.040112 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 21439
I20260924 21:14:50.040392 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.131:46207
--local_ip_for_outbound_sockets=127.6.210.131
--tserver_master_addrs=127.6.210.190:34097
--webserver_port=45451
--webserver_interface=127.6.210.131
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:43871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:14:50.043483 21575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:50.033967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.130:48521)
0924 21:14:50.034245 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:50.034321 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:50.035010 (+ 689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:50.035277 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:50.035280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:50.035538 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:14:50.035731 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:50.035736 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:50.036467 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:50.036468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:50.037412 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:50.037415 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:50.037841 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:50.037842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:50.037984 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:50.038611 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:14:50.038619 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:14:50.040158 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:14:50.042052 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:50.042058 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:50.042060 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:50.042414 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:50.042613 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:50.042614 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:50.042614 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:50.042762 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:50.042991 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:14:50.042995 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:14:50.043143 (+ 148us) client_negotiation.cc:769] Sending connection context
0924 21:14:50.043224 (+ 81us) client_negotiation.cc:240] Negotiation successful
0924 21:14:50.043343 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":117,"threads_started":1}
I20260924 21:14:50.043524 21437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:50.034080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:48521 (local address 127.6.210.190:34097)
0924 21:14:50.034158 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:50.034161 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:50.034424 (+ 263us) server_negotiation.cc:407] Connection header received
0924 21:14:50.035138 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:50.035140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:50.035157 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:50.035209 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:50.035844 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:50.036342 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:50.037514 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:50.037735 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:50.037830 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:50.040302 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:50.040304 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:50.040304 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:50.040313 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:50.041966 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:50.042491 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:50.042492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:50.042492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:50.042541 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:50.042832 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:50.042833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:50.042833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:50.042934 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:50.042983 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:50.043293 (+ 310us) server_negotiation.cc:299] Negotiation successful
0924 21:14:50.043394 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260924 21:14:50.043963 21573 heartbeater.cc:347] Connected to a master server at 127.6.210.190:34097
I20260924 21:14:50.044073 21573 heartbeater.cc:464] Registering TS with master...
I20260924 21:14:50.044265 21573 heartbeater.cc:511] Master 127.6.210.190:34097 requested a full tablet report, sending...
I20260924 21:14:50.044868 21246 ts_manager.cc:194] Registered new tserver with Master: 0993dfa60a2e4cef9e4246312d657aba (127.6.210.130:40125)
I20260924 21:14:50.045507 21246 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.130@KRBTEST.COM'} at 127.6.210.130:48521
I20260924 21:14:50.046221 21573 heartbeater.cc:503] Master 127.6.210.190:34097 was elected leader, sending a full tablet report...
Sep 24 21:14:50 dist-test-slave-q276 krb5kdc[20666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284489, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for kudu/127.6.210.130@KRBTEST.COM
I20260924 21:14:50.053570 21441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:50.045188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:40125 (local address 127.0.0.1:36372)
0924 21:14:50.045238 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:50.045247 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:50.045281 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:50.045558 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:50.045560 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:50.045967 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:14:50.046026 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:50.046028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:50.047123 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:50.047124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:50.048021 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:50.048024 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:50.048458 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:50.048459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:50.048529 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:50.049045 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:14:50.049051 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:14:50.050608 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:14:50.052341 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:50.052349 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:50.052351 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:50.052558 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:50.052793 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:50.052794 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:50.052795 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:50.052859 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:50.053220 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:14:50.053225 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:14:50.053268 (+ 43us) client_negotiation.cc:769] Sending connection context
0924 21:14:50.053360 (+ 92us) client_negotiation.cc:240] Negotiation successful
0924 21:14:50.053468 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260924 21:14:50.053599 21579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:50.045275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36372 (local address 127.6.210.130:40125)
0924 21:14:50.045408 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:50.045412 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:50.045421 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:14:50.045451 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:50.045454 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:50.045467 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:50.045532 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:50.046120 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:50.047044 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:50.048113 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:50.048384 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:50.048434 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:50.050699 (+ 2265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:50.050706 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:50.050707 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:50.050714 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:50.052237 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:50.052656 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:50.052659 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:50.052659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:50.052718 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:50.052954 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:50.052956 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:50.052956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:50.053147 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:50.053238 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:50.053407 (+ 169us) server_negotiation.cc:299] Negotiation successful
0924 21:14:50.053504 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":74,"threads_started":1}
I20260924 21:14:50.054011 21507 tablet_service.cc:1514] Processing DeleteTablet for tablet f64b56646fb7424388229f2bf3d109f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:14:49 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:36372
I20260924 21:14:50.054651 21580 tablet_replica.cc:330] stopping tablet replica
I20260924 21:14:50.054759 21580 raft_consensus.cc:2229] T f64b56646fb7424388229f2bf3d109f4 P 0993dfa60a2e4cef9e4246312d657aba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:14:50.054824 21580 raft_consensus.cc:2258] T f64b56646fb7424388229f2bf3d109f4 P 0993dfa60a2e4cef9e4246312d657aba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:14:50.055157 21580 ts_tablet_manager.cc:1905] T f64b56646fb7424388229f2bf3d109f4 P 0993dfa60a2e4cef9e4246312d657aba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:14:50.057305 21580 ts_tablet_manager.cc:1918] T f64b56646fb7424388229f2bf3d109f4 P 0993dfa60a2e4cef9e4246312d657aba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:14:50.057380 21580 log.cc:1196] T f64b56646fb7424388229f2bf3d109f4 P 0993dfa60a2e4cef9e4246312d657aba: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/wal/wals/f64b56646fb7424388229f2bf3d109f4
I20260924 21:14:50.057616 21580 ts_tablet_manager.cc:1939] T f64b56646fb7424388229f2bf3d109f4 P 0993dfa60a2e4cef9e4246312d657aba: Deleting consensus metadata
I20260924 21:14:50.058167 21232 catalog_manager.cc:4841] TS 0993dfa60a2e4cef9e4246312d657aba (127.6.210.130:40125): tablet f64b56646fb7424388229f2bf3d109f4 (table test-table [id=1fca48b83b374290a6faefb3ff9a60ec]) successfully deleted
W20260924 21:14:50.126582 21578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:14:50.126813 21578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:14:50.126850 21578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:14:50.128054 21578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:14:50.128121 21578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:14:50.128168 21578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.131
I20260924 21:14:50.129622 21578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:43871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.131:46207
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.210.131
--webserver_port=45451
--tserver_master_addrs=127.6.210.190:34097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.131
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:14:50.129889 21578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:14:50.130115 21578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:14:50.134985 21588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:50.135026 21585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:50.135285 21586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:14:50.135468 21578 server_base.cc:1034] running on GCE node
I20260924 21:14:50.135690 21578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:14:50.135946 21578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:14:50.137115 21578 hybrid_clock.cc:648] HybridClock initialized: now 1790284490137090 us; error 41 us; skew 500 ppm
Sep 24 21:14:50 dist-test-slave-q276 krb5kdc[20666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284490, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:14:50.139698 21578 init.cc:377] Logged in from keytab as kudu/127.6.210.131@KRBTEST.COM (short username kudu)
I20260924 21:14:50.140180 21578 webserver.cc:460] Webserver started at http://127.6.210.131:45451/ using document root <none> and password file <none>
I20260924 21:14:50.140419 21578 fs_manager.cc:359] Metadata directory not provided
I20260924 21:14:50.140511 21578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:14:50.142078 21578 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 21:14:50.142747 21595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:50.142920 21578 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:50.143023 21578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/wal
uuid: "f910de0a3bb54e69af8579664236d51b"
format_stamp: "Formatted at 2026-09-24 21:14:49 on dist-test-slave-q276"
server_key: "31e4d1bc6d022e31058502c8ac4546f9"
server_key_iv: "fb778e48bcd677f0158db473f801126d"
server_key_version: "encryptionkey@0"
I20260924 21:14:50.143396 21578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:14:50.156525 21578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:14:50.158406 21578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:14:50.158573 21578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:14:50.158806 21578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:14:50.159119 21578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:14:50.159173 21578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:50.159215 21578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:14:50.159242 21578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:50.164870 21578 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.131:46207
I20260924 21:14:50.164911 21708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.131:46207 every 8 connection(s)
I20260924 21:14:50.165251 21578 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
I20260924 21:14:50.166177 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 21578
Sep 24 21:14:50 dist-test-slave-q276 krb5kdc[20666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284490, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:14:50.173451 21713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:50.166564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46086 (local address 127.6.210.190:34097)
0924 21:14:50.166669 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:50.166671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:50.166680 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:14:50.166755 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:50.166755 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:50.166768 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:50.166826 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:50.167471 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:50.168030 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:50.168887 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:50.169078 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:50.169127 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:50.170673 (+ 1546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:50.170674 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:50.170675 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:50.170683 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:50.172259 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:50.172619 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:50.172621 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:50.172621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:50.172688 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:50.172894 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:50.172895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:50.172895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:50.173017 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:50.173206 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:50.173242 (+ 36us) server_negotiation.cc:299] Negotiation successful
0924 21:14:50.173330 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":57,"threads_started":1}
I20260924 21:14:50.175446 21437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:50.166016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:38131 (local address 127.6.210.190:34097)
0924 21:14:50.166102 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:50.166104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:50.166380 (+ 276us) server_negotiation.cc:407] Connection header received
0924 21:14:50.167003 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:50.167005 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:50.167015 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:50.167079 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:50.167956 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:50.168458 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:50.169752 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:50.169966 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:50.170050 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:50.172441 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:50.172443 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:50.172443 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:50.172450 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:50.173925 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:50.174499 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:50.174500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:50.174500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:50.174553 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:50.174833 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:50.174834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:50.174835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:50.174940 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:50.175014 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:50.175289 (+ 275us) server_negotiation.cc:299] Negotiation successful
0924 21:14:50.175365 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260924 21:14:50.175652 21711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:50.166015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.131:38131)
0924 21:14:50.166221 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:50.166290 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:50.166907 (+ 617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:50.167135 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:50.167138 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:50.167671 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:14:50.167859 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:50.167863 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:50.168552 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:50.168553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:50.169659 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:50.169661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:50.170061 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:50.170062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:50.170194 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:50.170701 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:14:50.170710 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:14:50.172330 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:14:50.174019 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:50.174025 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:50.174028 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:50.174368 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:50.174631 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:50.174631 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:50.174632 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:50.174761 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:50.175007 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:14:50.175013 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:14:50.175157 (+ 144us) client_negotiation.cc:769] Sending connection context
0924 21:14:50.175258 (+ 101us) client_negotiation.cc:240] Negotiation successful
0924 21:14:50.175408 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":132,"threads_started":1}
I20260924 21:14:50.176086 21709 heartbeater.cc:347] Connected to a master server at 127.6.210.190:34097
I20260924 21:14:50.176182 21709 heartbeater.cc:464] Registering TS with master...
I20260924 21:14:50.176313 21709 heartbeater.cc:511] Master 127.6.210.190:34097 requested a full tablet report, sending...
I20260924 21:14:50.176618 21246 ts_manager.cc:194] Registered new tserver with Master: f910de0a3bb54e69af8579664236d51b (127.6.210.131:46207)
I20260924 21:14:50.176926 21246 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.131@KRBTEST.COM'} at 127.6.210.131:38131
I20260924 21:14:50.185680 6986 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:14:50.188230 21437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:50.186037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46096 (local address 127.6.210.190:34097)
0924 21:14:50.186072 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:50.186074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:50.186176 (+ 102us) server_negotiation.cc:407] Connection header received
0924 21:14:50.186240 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:50.186241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:50.186328 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:50.186386 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:14:50.186471 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:50.186935 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:50.187647 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:50.187818 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:50.187866 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:50.187990 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:14:50.188080 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:14:50.188134 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:50.188164 (+ 30us) server_negotiation.cc:299] Negotiation successful
0924 21:14:50.188177 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 21:14:50.188489 21246 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:46096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:14:50.190078 21246 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:14:50.192214 21507 tablet_service.cc:1467] Processing CreateTablet for tablet 455681b4f2104a45991255bc23b1b7a1 (DEFAULT_TABLE table=test-table [id=4173f7bf766e41f48f706d82bea46392]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:14:50.192232 21363 tablet_service.cc:1467] Processing CreateTablet for tablet 455681b4f2104a45991255bc23b1b7a1 (DEFAULT_TABLE table=test-table [id=4173f7bf766e41f48f706d82bea46392]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:14:50.192408 21507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 455681b4f2104a45991255bc23b1b7a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:14:50.192408 21363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 455681b4f2104a45991255bc23b1b7a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:14:50.193703 21465 tablet_bootstrap.cc:492] T 455681b4f2104a45991255bc23b1b7a1 P 0993dfa60a2e4cef9e4246312d657aba: Bootstrap starting.
I20260924 21:14:50.193801 21326 tablet_bootstrap.cc:492] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0: Bootstrap starting.
I20260924 21:14:50.194054 21465 tablet_bootstrap.cc:654] T 455681b4f2104a45991255bc23b1b7a1 P 0993dfa60a2e4cef9e4246312d657aba: Neither blocks nor log segments found. Creating new log.
I20260924 21:14:50.194118 21326 tablet_bootstrap.cc:654] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0: Neither blocks nor log segments found. Creating new log.
I20260924 21:14:50.194664 21465 tablet_bootstrap.cc:492] T 455681b4f2104a45991255bc23b1b7a1 P 0993dfa60a2e4cef9e4246312d657aba: No bootstrap required, opened a new log
I20260924 21:14:50.194716 21465 ts_tablet_manager.cc:1397] T 455681b4f2104a45991255bc23b1b7a1 P 0993dfa60a2e4cef9e4246312d657aba: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260924 21:14:50.195047 21465 raft_consensus.cc:348] T 455681b4f2104a45991255bc23b1b7a1 P 0993dfa60a2e4cef9e4246312d657aba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910de0a3bb54e69af8579664236d51b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46207 } } peers { permanent_uuid: "602408af328941bcbd447c7246078ff0" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43583 } } peers { permanent_uuid: "0993dfa60a2e4cef9e4246312d657aba" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40125 } }
I20260924 21:14:50.195119 21465 raft_consensus.cc:374] T 455681b4f2104a45991255bc23b1b7a1 P 0993dfa60a2e4cef9e4246312d657aba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:14:50.195155 21465 raft_consensus.cc:729] T 455681b4f2104a45991255bc23b1b7a1 P 0993dfa60a2e4cef9e4246312d657aba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0993dfa60a2e4cef9e4246312d657aba, State: Initialized, Role: FOLLOWER
I20260924 21:14:50.195206 21465 consensus_queue.cc:260] T 455681b4f2104a45991255bc23b1b7a1 P 0993dfa60a2e4cef9e4246312d657aba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910de0a3bb54e69af8579664236d51b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46207 } } peers { permanent_uuid: "602408af328941bcbd447c7246078ff0" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43583 } } peers { permanent_uuid: "0993dfa60a2e4cef9e4246312d657aba" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40125 } }
I20260924 21:14:50.195310 21465 ts_tablet_manager.cc:1428] T 455681b4f2104a45991255bc23b1b7a1 P 0993dfa60a2e4cef9e4246312d657aba: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260924 21:14:50.197127 21441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:50.192039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:46207 (local address 127.0.0.1:60566)
0924 21:14:50.192087 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:50.192098 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:50.192131 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:50.193969 (+ 1838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:50.193970 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:50.193975 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:14:50.194023 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:50.194025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:50.195351 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:50.195352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:50.196272 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:50.196274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:50.196885 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:50.196886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:50.196937 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:50.196940 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:14:50.197023 (+ 83us) client_negotiation.cc:240] Negotiation successful
0924 21:14:50.197062 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260924 21:14:50.197165 21715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:50.192219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60566 (local address 127.6.210.131:46207)
0924 21:14:50.193785 (+ 1566us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:50.193791 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:50.193802 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:14:50.193847 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:50.193851 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:50.193869 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:50.193962 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:14:50.194090 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:50.195224 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:50.196354 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:50.196820 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:50.196873 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:50.196912 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:50.197043 (+ 131us) server_negotiation.cc:299] Negotiation successful
0924 21:14:50.197087 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1531,"thread_start_us":95,"threads_started":1}
I20260924 21:14:50.197254 21326 tablet_bootstrap.cc:492] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0: No bootstrap required, opened a new log
I20260924 21:14:50.197307 21326 ts_tablet_manager.cc:1397] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260924 21:14:50.197418 21326 raft_consensus.cc:348] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910de0a3bb54e69af8579664236d51b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46207 } } peers { permanent_uuid: "602408af328941bcbd447c7246078ff0" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43583 } } peers { permanent_uuid: "0993dfa60a2e4cef9e4246312d657aba" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40125 } }
I20260924 21:14:50.197485 21326 raft_consensus.cc:374] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:14:50.197539 21326 raft_consensus.cc:729] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602408af328941bcbd447c7246078ff0, State: Initialized, Role: FOLLOWER
I20260924 21:14:50.197607 21326 consensus_queue.cc:260] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910de0a3bb54e69af8579664236d51b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46207 } } peers { permanent_uuid: "602408af328941bcbd447c7246078ff0" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43583 } } peers { permanent_uuid: "0993dfa60a2e4cef9e4246312d657aba" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40125 } }
I20260924 21:14:50.197705 21326 ts_tablet_manager.cc:1428] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260924 21:14:50.197995 21643 tablet_service.cc:1467] Processing CreateTablet for tablet 455681b4f2104a45991255bc23b1b7a1 (DEFAULT_TABLE table=test-table [id=4173f7bf766e41f48f706d82bea46392]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:14:50.198417 21643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 455681b4f2104a45991255bc23b1b7a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:14:50.201135 21718 tablet_bootstrap.cc:492] T 455681b4f2104a45991255bc23b1b7a1 P f910de0a3bb54e69af8579664236d51b: Bootstrap starting.
I20260924 21:14:50.201831 21718 tablet_bootstrap.cc:654] T 455681b4f2104a45991255bc23b1b7a1 P f910de0a3bb54e69af8579664236d51b: Neither blocks nor log segments found. Creating new log.
I20260924 21:14:50.202432 21718 log.cc:824] T 455681b4f2104a45991255bc23b1b7a1 P f910de0a3bb54e69af8579664236d51b: Log is configured to *not* fsync() on all Append() calls
I20260924 21:14:50.203234 21718 tablet_bootstrap.cc:492] T 455681b4f2104a45991255bc23b1b7a1 P f910de0a3bb54e69af8579664236d51b: No bootstrap required, opened a new log
I20260924 21:14:50.203336 21718 ts_tablet_manager.cc:1397] T 455681b4f2104a45991255bc23b1b7a1 P f910de0a3bb54e69af8579664236d51b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:14:50.204700 21718 raft_consensus.cc:348] T 455681b4f2104a45991255bc23b1b7a1 P f910de0a3bb54e69af8579664236d51b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910de0a3bb54e69af8579664236d51b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46207 } } peers { permanent_uuid: "602408af328941bcbd447c7246078ff0" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43583 } } peers { permanent_uuid: "0993dfa60a2e4cef9e4246312d657aba" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40125 } }
I20260924 21:14:50.204864 21718 raft_consensus.cc:374] T 455681b4f2104a45991255bc23b1b7a1 P f910de0a3bb54e69af8579664236d51b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:14:50.204907 21718 raft_consensus.cc:729] T 455681b4f2104a45991255bc23b1b7a1 P f910de0a3bb54e69af8579664236d51b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f910de0a3bb54e69af8579664236d51b, State: Initialized, Role: FOLLOWER
I20260924 21:14:50.205046 21718 consensus_queue.cc:260] T 455681b4f2104a45991255bc23b1b7a1 P f910de0a3bb54e69af8579664236d51b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910de0a3bb54e69af8579664236d51b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46207 } } peers { permanent_uuid: "602408af328941bcbd447c7246078ff0" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43583 } } peers { permanent_uuid: "0993dfa60a2e4cef9e4246312d657aba" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40125 } }
I20260924 21:14:50.205314 21718 ts_tablet_manager.cc:1428] T 455681b4f2104a45991255bc23b1b7a1 P f910de0a3bb54e69af8579664236d51b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:14:50.205384 21709 heartbeater.cc:503] Master 127.6.210.190:34097 was elected leader, sending a full tablet report...
I20260924 21:14:50.249178 21440 raft_consensus.cc:482] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:14:50.249310 21440 raft_consensus.cc:504] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910de0a3bb54e69af8579664236d51b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46207 } } peers { permanent_uuid: "602408af328941bcbd447c7246078ff0" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43583 } } peers { permanent_uuid: "0993dfa60a2e4cef9e4246312d657aba" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40125 } }
I20260924 21:14:50.249617 21440 leader_election.cc:288] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f910de0a3bb54e69af8579664236d51b (127.6.210.131:46207), 0993dfa60a2e4cef9e4246312d657aba (127.6.210.130:40125)
I20260924 21:14:50.253566 21436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:50.249812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:46207 (local address 127.6.210.129:59843)
0924 21:14:50.249880 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:50.249893 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:50.249956 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:50.250228 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:50.250228 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:50.250230 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:14:50.250632 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:50.250634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:50.251428 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:50.251428 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:50.252530 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:50.252533 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:50.253242 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:50.253242 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:50.253347 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:50.253352 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 21:14:50.253419 (+ 67us) client_negotiation.cc:240] Negotiation successful
0924 21:14:50.253481 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":6}
I20260924 21:14:50.253647 21579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:50.249899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:47737 (local address 127.6.210.130:40125)
0924 21:14:50.249946 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:50.249948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:50.250183 (+ 235us) server_negotiation.cc:407] Connection header received
0924 21:14:50.250212 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:50.250212 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:50.250239 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:50.250321 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:14:50.250464 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:50.251188 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:50.252621 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:50.253247 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:50.253336 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:50.253413 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:50.253524 (+ 111us) server_negotiation.cc:299] Negotiation successful
0924 21:14:50.253566 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 21:14:50.253741 21715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:50.250001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:59843 (local address 127.6.210.131:46207)
0924 21:14:50.250074 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:50.250077 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:50.250088 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:14:50.250123 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:50.250123 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:50.250150 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:50.250208 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:14:50.250710 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:50.251361 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:50.252619 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:50.253161 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:50.253242 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:50.253251 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:50.253433 (+ 182us) server_negotiation.cc:299] Negotiation successful
0924 21:14:50.253477 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 21:14:50.254040 21721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:50.249813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:40125 (local address 127.6.210.129:47737)
0924 21:14:50.250090 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:50.250098 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:50.250143 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:50.250325 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:50.250326 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:50.250328 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:14:50.250390 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:50.250392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:50.251270 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:50.251270 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:50.252530 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:50.252533 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:50.253354 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:50.253355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:50.253459 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:50.253462 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:14:50.253513 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 21:14:50.253541 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":112,"thread_start_us":124,"threads_started":1}
I20260924 21:14:50.254029 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "455681b4f2104a45991255bc23b1b7a1" candidate_uuid: "602408af328941bcbd447c7246078ff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f910de0a3bb54e69af8579664236d51b" is_pre_election: true
I20260924 21:14:50.254202 21663 raft_consensus.cc:2454] T 455681b4f2104a45991255bc23b1b7a1 P f910de0a3bb54e69af8579664236d51b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 602408af328941bcbd447c7246078ff0 in term 0.
I20260924 21:14:50.254401 21527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "455681b4f2104a45991255bc23b1b7a1" candidate_uuid: "602408af328941bcbd447c7246078ff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0993dfa60a2e4cef9e4246312d657aba" is_pre_election: true
I20260924 21:14:50.254458 21321 leader_election.cc:302] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602408af328941bcbd447c7246078ff0, f910de0a3bb54e69af8579664236d51b; no voters:
I20260924 21:14:50.254590 21527 raft_consensus.cc:2454] T 455681b4f2104a45991255bc23b1b7a1 P 0993dfa60a2e4cef9e4246312d657aba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 602408af328941bcbd447c7246078ff0 in term 0.
I20260924 21:14:50.254587 21440 raft_consensus.cc:2781] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:14:50.254635 21440 raft_consensus.cc:482] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:14:50.254658 21440 raft_consensus.cc:3037] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:14:50.255381 21440 raft_consensus.cc:504] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910de0a3bb54e69af8579664236d51b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46207 } } peers { permanent_uuid: "602408af328941bcbd447c7246078ff0" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43583 } } peers { permanent_uuid: "0993dfa60a2e4cef9e4246312d657aba" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40125 } }
I20260924 21:14:50.255548 21440 leader_election.cc:288] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [CANDIDATE]: Term 1 election: Requested vote from peers f910de0a3bb54e69af8579664236d51b (127.6.210.131:46207), 0993dfa60a2e4cef9e4246312d657aba (127.6.210.130:40125)
I20260924 21:14:50.255690 21663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "455681b4f2104a45991255bc23b1b7a1" candidate_uuid: "602408af328941bcbd447c7246078ff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f910de0a3bb54e69af8579664236d51b"
I20260924 21:14:50.255738 21527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "455681b4f2104a45991255bc23b1b7a1" candidate_uuid: "602408af328941bcbd447c7246078ff0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0993dfa60a2e4cef9e4246312d657aba"
I20260924 21:14:50.255774 21663 raft_consensus.cc:3037] T 455681b4f2104a45991255bc23b1b7a1 P f910de0a3bb54e69af8579664236d51b [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:14:50.255805 21527 raft_consensus.cc:3037] T 455681b4f2104a45991255bc23b1b7a1 P 0993dfa60a2e4cef9e4246312d657aba [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:14:50.256641 21663 raft_consensus.cc:2454] T 455681b4f2104a45991255bc23b1b7a1 P f910de0a3bb54e69af8579664236d51b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602408af328941bcbd447c7246078ff0 in term 1.
I20260924 21:14:50.256641 21527 raft_consensus.cc:2454] T 455681b4f2104a45991255bc23b1b7a1 P 0993dfa60a2e4cef9e4246312d657aba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 602408af328941bcbd447c7246078ff0 in term 1.
I20260924 21:14:50.256801 21321 leader_election.cc:302] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 602408af328941bcbd447c7246078ff0, f910de0a3bb54e69af8579664236d51b; no voters:
I20260924 21:14:50.256910 21440 raft_consensus.cc:2781] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:14:50.257073 21440 raft_consensus.cc:686] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [term 1 LEADER]: Becoming Leader. State: Replica: 602408af328941bcbd447c7246078ff0, State: Running, Role: LEADER
I20260924 21:14:50.257156 21440 consensus_queue.cc:237] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910de0a3bb54e69af8579664236d51b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46207 } } peers { permanent_uuid: "602408af328941bcbd447c7246078ff0" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43583 } } peers { permanent_uuid: "0993dfa60a2e4cef9e4246312d657aba" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40125 } }
I20260924 21:14:50.257910 21246 catalog_manager.cc:5495] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 602408af328941bcbd447c7246078ff0 (127.6.210.129). New cstate: current_term: 1 leader_uuid: "602408af328941bcbd447c7246078ff0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910de0a3bb54e69af8579664236d51b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "602408af328941bcbd447c7246078ff0" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43583 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0993dfa60a2e4cef9e4246312d657aba" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40125 } health_report { overall_health: UNKNOWN } } }
I20260924 21:14:50.270216 21442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:50.268076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60198 (local address 127.6.210.129:43583)
0924 21:14:50.268136 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:50.268138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:50.268147 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:14:50.268205 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:50.268206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:50.268222 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:50.268280 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:14:50.268387 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:50.268838 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:50.269550 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:50.269727 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:50.269778 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:50.269893 (+ 115us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:14:50.269997 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:14:50.270045 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:50.270100 (+ 55us) server_negotiation.cc:299] Negotiation successful
0924 21:14:50.270117 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 21:14:50.271692 21663 raft_consensus.cc:1261] T 455681b4f2104a45991255bc23b1b7a1 P f910de0a3bb54e69af8579664236d51b [term 1 FOLLOWER]: Refusing update from remote peer 602408af328941bcbd447c7246078ff0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:14:50.271737 21527 raft_consensus.cc:1261] T 455681b4f2104a45991255bc23b1b7a1 P 0993dfa60a2e4cef9e4246312d657aba [term 1 FOLLOWER]: Refusing update from remote peer 602408af328941bcbd447c7246078ff0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:14:50.272032 21440 consensus_queue.cc:1035] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f910de0a3bb54e69af8579664236d51b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:14:50.272187 21722 consensus_queue.cc:1035] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0993dfa60a2e4cef9e4246312d657aba" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:14:50.274816 21726 mvcc.cc:205] Tried to move back new op lower bound from 7333005272150556672 to 7333005272093880320. Current Snapshot: MvccSnapshot[applied={T|T < 7333005272150556672 or (T in {7333005272150556672})}]
I20260924 21:14:50.276381 21245 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:46096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:14:50.276485 21245 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:46096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:14:50.277199 21245 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb: Sending DeleteTablet for 3 replicas of tablet 455681b4f2104a45991255bc23b1b7a1
I20260924 21:14:50.277499 21643 tablet_service.cc:1514] Processing DeleteTablet for tablet 455681b4f2104a45991255bc23b1b7a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:14:50 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:60566
I20260924 21:14:50.277601 21507 tablet_service.cc:1514] Processing DeleteTablet for tablet 455681b4f2104a45991255bc23b1b7a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:14:50 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:36372
I20260924 21:14:50.277702 21363 tablet_service.cc:1514] Processing DeleteTablet for tablet 455681b4f2104a45991255bc23b1b7a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:14:50 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:60196
I20260924 21:14:50.277740 21735 tablet_replica.cc:330] T 455681b4f2104a45991255bc23b1b7a1 P f910de0a3bb54e69af8579664236d51b: stopping tablet replica
I20260924 21:14:50.277788 21580 tablet_replica.cc:330] T 455681b4f2104a45991255bc23b1b7a1 P 0993dfa60a2e4cef9e4246312d657aba: stopping tablet replica
I20260924 21:14:50.277841 21735 raft_consensus.cc:2229] T 455681b4f2104a45991255bc23b1b7a1 P f910de0a3bb54e69af8579664236d51b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:14:50.277846 21580 raft_consensus.cc:2229] T 455681b4f2104a45991255bc23b1b7a1 P 0993dfa60a2e4cef9e4246312d657aba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:14:50.277908 21580 raft_consensus.cc:2258] T 455681b4f2104a45991255bc23b1b7a1 P 0993dfa60a2e4cef9e4246312d657aba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:14:50.277951 21735 raft_consensus.cc:2258] T 455681b4f2104a45991255bc23b1b7a1 P f910de0a3bb54e69af8579664236d51b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:14:50.277985 21443 tablet_replica.cc:330] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0: stopping tablet replica
I20260924 21:14:50.278041 21443 raft_consensus.cc:2229] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:14:50.278117 21443 raft_consensus.cc:2258] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:14:50.278431 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 21285
I20260924 21:14:50.278532 21443 ts_tablet_manager.cc:1905] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:14:50.278616 21580 ts_tablet_manager.cc:1905] T 455681b4f2104a45991255bc23b1b7a1 P 0993dfa60a2e4cef9e4246312d657aba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:14:50.280373 21580 ts_tablet_manager.cc:1918] T 455681b4f2104a45991255bc23b1b7a1 P 0993dfa60a2e4cef9e4246312d657aba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:14:50.280440 21580 log.cc:1196] T 455681b4f2104a45991255bc23b1b7a1 P 0993dfa60a2e4cef9e4246312d657aba: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/wal/wals/455681b4f2104a45991255bc23b1b7a1
I20260924 21:14:50.280522 21735 ts_tablet_manager.cc:1905] T 455681b4f2104a45991255bc23b1b7a1 P f910de0a3bb54e69af8579664236d51b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:14:50.280654 21580 ts_tablet_manager.cc:1939] T 455681b4f2104a45991255bc23b1b7a1 P 0993dfa60a2e4cef9e4246312d657aba: Deleting consensus metadata
I20260924 21:14:50.281054 21232 catalog_manager.cc:4841] TS 0993dfa60a2e4cef9e4246312d657aba (127.6.210.130:40125): tablet 455681b4f2104a45991255bc23b1b7a1 (table test-table [id=4173f7bf766e41f48f706d82bea46392]) successfully deleted
I20260924 21:14:50.281889 21735 ts_tablet_manager.cc:1918] T 455681b4f2104a45991255bc23b1b7a1 P f910de0a3bb54e69af8579664236d51b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:14:50.281980 21735 log.cc:1196] T 455681b4f2104a45991255bc23b1b7a1 P f910de0a3bb54e69af8579664236d51b: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/wal/wals/455681b4f2104a45991255bc23b1b7a1
I20260924 21:14:50.282230 21735 ts_tablet_manager.cc:1939] T 455681b4f2104a45991255bc23b1b7a1 P f910de0a3bb54e69af8579664236d51b: Deleting consensus metadata
I20260924 21:14:50.282665 21231 catalog_manager.cc:4841] TS f910de0a3bb54e69af8579664236d51b (127.6.210.131:46207): tablet 455681b4f2104a45991255bc23b1b7a1 (table test-table [id=4173f7bf766e41f48f706d82bea46392]) successfully deleted
W20260924 21:14:50.284253 21230 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.210.129:43583 (error 108)
W20260924 21:14:50.284826 21441 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:14:50.284586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:43583 (local address 127.0.0.1:60202)
0924 21:14:50.284675 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:50.284688 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:50.284741 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:50.284781 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.129:43583: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":37}
I20260924 21:14:50.285087 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 21439
W20260924 21:14:50.284943 21230 catalog_manager.cc:4584] TS 602408af328941bcbd447c7246078ff0 (127.6.210.129:43583): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 455681b4f2104a45991255bc23b1b7a1: Network error: Client connection negotiation failed: client connection to 127.6.210.129:43583: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260924 21:14:50.290094 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 21578
I20260924 21:14:50.294642 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 21213
I20260924 21:14:50.299764 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:34097
--webserver_interface=127.6.210.190
--webserver_port=44911
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:43871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:34097
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:14:50.380330 21737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:14:50.380573 21737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:14:50.380612 21737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:14:50.381778 21737 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:14:50.381843 21737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:14:50.381870 21737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:14:50.381898 21737 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:14:50.381929 21737 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:14:50.383443 21737 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:43871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:34097
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:34097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=44911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:14:50.383721 21737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:14:50.383950 21737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:14:50.388753 21742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:50.388769 21743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:14:50.388938 21737 server_base.cc:1034] running on GCE node
W20260924 21:14:50.388760 21745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:14:50.389263 21737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:14:50.389564 21737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:14:50.390704 21737 hybrid_clock.cc:648] HybridClock initialized: now 1790284490390688 us; error 36 us; skew 500 ppm
Sep 24 21:14:50 dist-test-slave-q276 krb5kdc[20666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284490, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:14:50.393239 21737 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:14:50.393705 21737 webserver.cc:460] Webserver started at http://127.6.210.190:44911/ using document root <none> and password file <none>
I20260924 21:14:50.393889 21737 fs_manager.cc:359] Metadata directory not provided
I20260924 21:14:50.393934 21737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:14:50.395256 21737 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:14:50.395867 21752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:50.396032 21737 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:14:50.396134 21737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "e5faabaa1eee4d84a3504cde1f09eedb"
format_stamp: "Formatted at 2026-09-24 21:14:48 on dist-test-slave-q276"
server_key: "20b16c68c5ce2f11c4de44030da35020"
server_key_iv: "278bc3738ccdbfd86105d88b71a77ccd"
server_key_version: "encryptionkey@0"
I20260924 21:14:50.396468 21737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:14:50.408568 21737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:14:50.410529 21737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:14:50.410688 21737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:14:50.414666 21737 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:34097
I20260924 21:14:50.414702 21804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:34097 every 8 connection(s)
I20260924 21:14:50.415081 21737 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:14:50.416579 21805 sys_catalog.cc:263] Verifying existing consensus state
I20260924 21:14:50.417234 21805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb: Bootstrap starting.
I20260924 21:14:50.424374 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 21737
I20260924 21:14:50.424819 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.129:43583
--local_ip_for_outbound_sockets=127.6.210.129
--tserver_master_addrs=127.6.210.190:34097
--webserver_port=42583
--webserver_interface=127.6.210.129
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:43871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:14:50.425596 21805 log.cc:824] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb: Log is configured to *not* fsync() on all Append() calls
I20260924 21:14:50.428403 21805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 21:14:50.428719 21805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb: Bootstrap complete.
I20260924 21:14:50.431034 21805 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5faabaa1eee4d84a3504cde1f09eedb" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34097 } }
I20260924 21:14:50.431253 21805 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5faabaa1eee4d84a3504cde1f09eedb, State: Initialized, Role: FOLLOWER
I20260924 21:14:50.431423 21805 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5faabaa1eee4d84a3504cde1f09eedb" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34097 } }
I20260924 21:14:50.431491 21805 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:14:50.431583 21805 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:14:50.431643 21805 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [term 2 FOLLOWER]: Advancing to term 3
I20260924 21:14:50.432664 21805 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5faabaa1eee4d84a3504cde1f09eedb" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34097 } }
I20260924 21:14:50.432750 21805 leader_election.cc:302] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5faabaa1eee4d84a3504cde1f09eedb; no voters:
I20260924 21:14:50.432971 21805 leader_election.cc:288] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [CANDIDATE]: Term 3 election: Requested vote from peers
I20260924 21:14:50.433019 21809 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [term 3 FOLLOWER]: Leader election won for term 3
I20260924 21:14:50.433148 21809 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [term 3 LEADER]: Becoming Leader. State: Replica: e5faabaa1eee4d84a3504cde1f09eedb, State: Running, Role: LEADER
I20260924 21:14:50.433246 21805 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:14:50.433246 21809 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5faabaa1eee4d84a3504cde1f09eedb" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34097 } }
I20260924 21:14:50.433568 21810 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5faabaa1eee4d84a3504cde1f09eedb. Latest consensus state: current_term: 3 leader_uuid: "e5faabaa1eee4d84a3504cde1f09eedb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5faabaa1eee4d84a3504cde1f09eedb" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34097 } } }
I20260924 21:14:50.433665 21810 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [sys.catalog]: This master's current role is: LEADER
I20260924 21:14:50.433565 21811 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "e5faabaa1eee4d84a3504cde1f09eedb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5faabaa1eee4d84a3504cde1f09eedb" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34097 } } }
I20260924 21:14:50.433847 21811 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5faabaa1eee4d84a3504cde1f09eedb [sys.catalog]: This master's current role is: LEADER
I20260924 21:14:50.433867 21814 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:14:50.434882 21814 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:14:50.435012 21814 catalog_manager.cc:1259] Loaded cluster ID: cf8ab8b0807d453abf05c7b382fd51e9
I20260924 21:14:50.435217 21814 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260924 21:14:50.435462 21814 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f65fb38b9ad google::LogMessage::Fail() at ??:0
@ 0x7f65fb38e88a google::LogMessage::SendToLog() at ??:0
@ 0x7f65fb38b4de google::LogMessage::Flush() at ??:0
@ 0x7f65fb38ca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f66032b00f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f66032b98b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f65fd2590d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f65fd24e954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f65fe0b26db start_thread at ??:0
@ 0x7f65f9e8961f clone at ??:0
W20260924 21:14:50.521215 21808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:14:50.521412 21808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:14:50.521437 21808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:14:50.522667 21808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:14:50.522709 21808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:14:50.522737 21808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.129
I20260924 21:14:50.524281 21808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:43871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.129:43583
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.210.129
--webserver_port=42583
--tserver_master_addrs=127.6.210.190:34097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.129
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:14:50.524545 21808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:14:50.524791 21808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:14:50.529910 21842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:50.529922 21841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:50.529920 21844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:14:50.530308 21808 server_base.cc:1034] running on GCE node
I20260924 21:14:50.530510 21808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:14:50.530763 21808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:14:50.531931 21808 hybrid_clock.cc:648] HybridClock initialized: now 1790284490531918 us; error 30 us; skew 500 ppm
Sep 24 21:14:50 dist-test-slave-q276 krb5kdc[20666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284490, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:14:50.534461 21808 init.cc:377] Logged in from keytab as kudu/127.6.210.129@KRBTEST.COM (short username kudu)
I20260924 21:14:50.534943 21808 webserver.cc:460] Webserver started at http://127.6.210.129:42583/ using document root <none> and password file <none>
I20260924 21:14:50.535194 21808 fs_manager.cc:359] Metadata directory not provided
I20260924 21:14:50.535260 21808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:14:50.536717 21808 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:14:50.537492 21851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:50.537715 21808 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:14:50.537822 21808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/wal
uuid: "602408af328941bcbd447c7246078ff0"
format_stamp: "Formatted at 2026-09-24 21:14:49 on dist-test-slave-q276"
server_key: "4bd0f6f357f1beb76fdd68011a669b3d"
server_key_iv: "3094b15333c4ea467092867d3661a6a5"
server_key_version: "encryptionkey@0"
I20260924 21:14:50.538165 21808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:14:50.575923 21808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:14:50.578171 21808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:14:50.578358 21808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:14:50.578608 21808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:14:50.579070 21858 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 21:14:50.581023 21808 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 21:14:50.581084 21808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260924 21:14:50.581127 21808 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 21:14:50.581743 21808 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 21:14:50.581799 21808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260924 21:14:50.581884 21858 tablet_bootstrap.cc:492] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0: Bootstrap starting.
I20260924 21:14:50.583956 21858 log.cc:824] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0: Log is configured to *not* fsync() on all Append() calls
I20260924 21:14:50.586838 21858 tablet_bootstrap.cc:492] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 21:14:50.587144 21858 tablet_bootstrap.cc:492] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0: Bootstrap complete.
I20260924 21:14:50.587303 21858 ts_tablet_manager.cc:1397] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260924 21:14:50.588971 21808 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.129:43583
I20260924 21:14:50.589072 21966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.129:43583 every 8 connection(s)
I20260924 21:14:50.589342 21808 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
I20260924 21:14:50.589170 21858 raft_consensus.cc:348] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910de0a3bb54e69af8579664236d51b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46207 } } peers { permanent_uuid: "602408af328941bcbd447c7246078ff0" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43583 } } peers { permanent_uuid: "0993dfa60a2e4cef9e4246312d657aba" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40125 } }
I20260924 21:14:50.589514 21858 raft_consensus.cc:729] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602408af328941bcbd447c7246078ff0, State: Initialized, Role: FOLLOWER
I20260924 21:14:50.589825 21858 consensus_queue.cc:260] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910de0a3bb54e69af8579664236d51b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46207 } } peers { permanent_uuid: "602408af328941bcbd447c7246078ff0" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43583 } } peers { permanent_uuid: "0993dfa60a2e4cef9e4246312d657aba" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40125 } }
I20260924 21:14:50.590102 21858 ts_tablet_manager.cc:1428] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260924 21:14:50.591167 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 21808
I20260924 21:14:50.591434 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.130:40125
--local_ip_for_outbound_sockets=127.6.210.130
--tserver_master_addrs=127.6.210.190:34097
--webserver_port=46845
--webserver_interface=127.6.210.130
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:43871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:14:50.629769 21970 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:14:50.590056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.129:35505)
0924 21:14:50.590252 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:50.590312 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:50.590458 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:50.629568 (+ 39110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":88,"threads_started":1}
W20260924 21:14:50.630059 21967 heartbeater.cc:644] Failed to heartbeat to 127.6.210.190:34097 (0 consecutive failures): Network error: Failed to ping master at 127.6.210.190:34097: Client connection negotiation failed: client connection to 127.6.210.190:34097: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 21:14:50.630443 21970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:50.630323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.129:34343)
0924 21:14:50.630379 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:50.630417 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260924 21:14:50.630779 21970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:50.630695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.129:46085)
0924 21:14:50.630739 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:50.630762 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260924 21:14:50.630903 21967 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 21:14:50.674603 21971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:14:50.674850 21971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:14:50.674891 21971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:14:50.676045 21971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:14:50.676116 21971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:14:50.676163 21971 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.130
I20260924 21:14:50.677616 21971 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:43871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.130:40125
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.210.130
--webserver_port=46845
--tserver_master_addrs=127.6.210.190:34097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.130
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:14:50.677874 21971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:14:50.678102 21971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:14:50.683051 21976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:50.683028 21979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:50.683099 21977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:14:50.683288 21971 server_base.cc:1034] running on GCE node
I20260924 21:14:50.683532 21971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:14:50.683769 21971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:14:50.684908 21971 hybrid_clock.cc:648] HybridClock initialized: now 1790284490684885 us; error 46 us; skew 500 ppm
Sep 24 21:14:50 dist-test-slave-q276 krb5kdc[20666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284490, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:14:50.687577 21971 init.cc:377] Logged in from keytab as kudu/127.6.210.130@KRBTEST.COM (short username kudu)
I20260924 21:14:50.688058 21971 webserver.cc:460] Webserver started at http://127.6.210.130:46845/ using document root <none> and password file <none>
I20260924 21:14:50.688275 21971 fs_manager.cc:359] Metadata directory not provided
I20260924 21:14:50.688320 21971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:14:50.689606 21971 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:14:50.690166 21986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:50.690383 21971 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:50.690469 21971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/wal
uuid: "0993dfa60a2e4cef9e4246312d657aba"
format_stamp: "Formatted at 2026-09-24 21:14:49 on dist-test-slave-q276"
server_key: "a62cfb61c93c82765336407613e8ade0"
server_key_iv: "26fa3baf3013f98560b86430e6080281"
server_key_version: "encryptionkey@0"
I20260924 21:14:50.690799 21971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:14:50.707178 21971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:14:50.709576 21971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:14:50.709690 21971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:14:50.709883 21971 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:14:50.710199 21971 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:14:50.710235 21971 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:50.710299 21971 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:14:50.710321 21971 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:50.716216 21971 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.130:40125
I20260924 21:14:50.716265 22099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.130:40125 every 8 connection(s)
I20260924 21:14:50.716607 21971 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
I20260924 21:14:50.717597 22102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:50.717222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.130:43975)
0924 21:14:50.717377 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:50.717444 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":74,"threads_started":1}
W20260924 21:14:50.717986 22100 heartbeater.cc:644] Failed to heartbeat to 127.6.210.190:34097 (0 consecutive failures): Network error: Failed to ping master at 127.6.210.190:34097: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
I20260924 21:14:50.718318 22102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:50.718167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.130:43589)
0924 21:14:50.718218 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:50.718250 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260924 21:14:50.718606 22102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:50.718520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.130:57641)
0924 21:14:50.718564 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:50.718587 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260924 21:14:50.718709 22100 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 21:14:50.726434 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 21971
I20260924 21:14:50.726637 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.131:46207
--local_ip_for_outbound_sockets=127.6.210.131
--tserver_master_addrs=127.6.210.190:34097
--webserver_port=45451
--webserver_interface=127.6.210.131
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:43871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:14:50.807626 22103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:14:50.807827 22103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:14:50.807850 22103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:14:50.808991 22103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:14:50.809033 22103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:14:50.809059 22103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.131
I20260924 21:14:50.810451 22103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:43871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.131:46207
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.210.131
--webserver_port=45451
--tserver_master_addrs=127.6.210.190:34097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.131
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:14:50.810668 22103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:14:50.810874 22103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:14:50.815714 22109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:50.815748 22111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:14:50.815737 22108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:14:50.815934 22103 server_base.cc:1034] running on GCE node
I20260924 21:14:50.816182 22103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:14:50.816425 22103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:14:50.817615 22103 hybrid_clock.cc:648] HybridClock initialized: now 1790284490817602 us; error 37 us; skew 500 ppm
Sep 24 21:14:50 dist-test-slave-q276 krb5kdc[20666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284490, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:14:50.820180 22103 init.cc:377] Logged in from keytab as kudu/127.6.210.131@KRBTEST.COM (short username kudu)
I20260924 21:14:50.820607 22103 webserver.cc:460] Webserver started at http://127.6.210.131:45451/ using document root <none> and password file <none>
I20260924 21:14:50.820830 22103 fs_manager.cc:359] Metadata directory not provided
I20260924 21:14:50.820901 22103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:14:50.822237 22103 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:14:50.822891 22118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:50.823062 22103 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:50.823163 22103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/wal
uuid: "f910de0a3bb54e69af8579664236d51b"
format_stamp: "Formatted at 2026-09-24 21:14:49 on dist-test-slave-q276"
server_key: "31e4d1bc6d022e31058502c8ac4546f9"
server_key_iv: "fb778e48bcd677f0158db473f801126d"
server_key_version: "encryptionkey@0"
I20260924 21:14:50.823529 22103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:14:50.845293 22103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:14:50.847873 22103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:14:50.848065 22103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:14:50.848326 22103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:14:50.848659 22103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:14:50.848716 22103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:50.848757 22103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:14:50.848788 22103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:14:50.854528 22103 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.131:46207
I20260924 21:14:50.854588 22231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.131:46207 every 8 connection(s)
I20260924 21:14:50.854964 22103 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
I20260924 21:14:50.856132 22234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:50.855603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.131:36119)
0924 21:14:50.855817 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:50.855972 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":79,"threads_started":1}
W20260924 21:14:50.856580 22232 heartbeater.cc:644] Failed to heartbeat to 127.6.210.190:34097 (0 consecutive failures): Network error: Failed to ping master at 127.6.210.190:34097: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
I20260924 21:14:50.856930 22234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:50.856790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.131:41455)
0924 21:14:50.856835 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:50.856911 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260924 21:14:50.857209 22234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:50.857136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.131:34961)
0924 21:14:50.857175 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:50.857195 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260924 21:14:50.857337 22232 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 21:14:50.861588 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 22103
I20260924 21:14:51.631568 22236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:51.631330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.129:48801)
0924 21:14:51.631461 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:51.631507 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":69,"threads_started":1}
I20260924 21:14:51.719249 22237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:51.719055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.130:49229)
0924 21:14:51.719168 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:51.719199 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":57,"threads_started":1}
I20260924 21:14:51.857904 22238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:51.857708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.131:48467)
0924 21:14:51.857818 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:51.857854 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":56,"threads_started":1}
I20260924 21:14:52.045575 22239 raft_consensus.cc:482] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:14:52.045707 22239 raft_consensus.cc:504] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910de0a3bb54e69af8579664236d51b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46207 } } peers { permanent_uuid: "602408af328941bcbd447c7246078ff0" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43583 } } peers { permanent_uuid: "0993dfa60a2e4cef9e4246312d657aba" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40125 } }
I20260924 21:14:52.046353 22239 leader_election.cc:288] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f910de0a3bb54e69af8579664236d51b (127.6.210.131:46207), 0993dfa60a2e4cef9e4246312d657aba (127.6.210.130:40125)
Sep 24 21:14:52 dist-test-slave-q276 krb5kdc[20666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284490, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for kudu/127.6.210.131@KRBTEST.COM
Sep 24 21:14:52 dist-test-slave-q276 krb5kdc[20666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284490, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for kudu/127.6.210.130@KRBTEST.COM
I20260924 21:14:52.058167 22236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:52.046493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:46207 (local address 127.6.210.129:41761)
0924 21:14:52.046558 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:52.046571 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:52.046656 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:52.047528 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:52.047531 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:52.047943 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:14:52.048178 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:52.048184 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:52.049417 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:52.049418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:52.050539 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:52.050542 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:52.051144 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:52.051145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:52.051259 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:52.051916 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:14:52.051920 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:14:52.053680 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:14:52.056125 (+ 2445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:52.056133 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:52.056135 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:52.056586 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:52.056881 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:52.056882 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:52.056882 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:52.056982 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:52.057505 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:14:52.057511 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:14:52.057690 (+ 179us) client_negotiation.cc:769] Sending connection context
0924 21:14:52.057821 (+ 131us) client_negotiation.cc:240] Negotiation successful
0924 21:14:52.058039 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":16}
I20260924 21:14:52.058190 22241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:52.046660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:34981 (local address 127.6.210.130:40125)
0924 21:14:52.046799 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:52.046810 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:52.046857 (+ 47us) server_negotiation.cc:407] Connection header received
0924 21:14:52.047300 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:52.047302 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:52.047347 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:52.047506 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:52.048480 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:52.049659 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:52.050622 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:52.051038 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:52.051152 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:52.054252 (+ 3100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:52.054258 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:52.054258 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:52.054309 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:52.056165 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:52.056891 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:52.056893 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:52.056893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:52.056971 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:52.057171 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:52.057172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:52.057172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:52.057426 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:52.057543 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:52.057934 (+ 391us) server_negotiation.cc:299] Negotiation successful
0924 21:14:52.058078 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":57,"threads_started":1}
I20260924 21:14:52.058194 22242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:52.046671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:41761 (local address 127.6.210.131:46207)
0924 21:14:52.046839 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0924 21:14:52.046847 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 21:14:52.046874 (+ 27us) server_negotiation.cc:407] Connection header received
0924 21:14:52.047292 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:14:52.047294 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:14:52.047334 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:14:52.047518 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:14:52.048317 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:52.049249 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:52.050654 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:14:52.051042 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:14:52.051166 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:52.053954 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:14:52.053964 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:14:52.053964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:14:52.053975 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:14:52.056038 (+ 2063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:52.056715 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:52.056718 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:52.056718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:52.056806 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:14:52.057089 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:14:52.057089 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:14:52.057090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:14:52.057426 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:14:52.057519 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0924 21:14:52.057934 (+ 415us) server_negotiation.cc:299] Negotiation successful
0924 21:14:52.058078 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":58,"threads_started":1}
I20260924 21:14:52.058238 22240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:52.046514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:40125 (local address 127.6.210.129:34981)
0924 21:14:52.046668 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:52.046680 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:14:52.046740 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:14:52.047532 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:14:52.047533 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:14:52.047956 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:14:52.048364 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:52.048366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:52.049770 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:52.049771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:52.050538 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:14:52.050541 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:14:52.051130 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:14:52.051131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:14:52.051259 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:14:52.051874 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:14:52.051882 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:14:52.054139 (+ 2257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:14:52.056244 (+ 2105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:52.056244 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:52.056245 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:52.056586 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:52.057041 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:14:52.057041 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:14:52.057042 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:14:52.057100 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:14:52.057527 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:14:52.057527 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:14:52.057685 (+ 158us) client_negotiation.cc:769] Sending connection context
0924 21:14:52.057821 (+ 136us) client_negotiation.cc:240] Negotiation successful
0924 21:14:52.058092 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":149,"thread_start_us":74,"threads_started":1}
W20260924 21:14:52.059415 21854 leader_election.cc:341] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0993dfa60a2e4cef9e4246312d657aba (127.6.210.130:40125): Not found: Tablet not found: 455681b4f2104a45991255bc23b1b7a1
W20260924 21:14:52.059530 21853 leader_election.cc:341] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f910de0a3bb54e69af8579664236d51b (127.6.210.131:46207): Not found: Tablet not found: 455681b4f2104a45991255bc23b1b7a1
I20260924 21:14:52.059580 21853 leader_election.cc:302] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 602408af328941bcbd447c7246078ff0; no voters: 0993dfa60a2e4cef9e4246312d657aba, f910de0a3bb54e69af8579664236d51b
I20260924 21:14:52.059686 22239 raft_consensus.cc:2726] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 21:14:52.632323 22243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:52.632076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.129:48505)
0924 21:14:52.632229 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:52.632274 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":62,"threads_started":1}
I20260924 21:14:52.719985 22244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:52.719782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.130:41471)
0924 21:14:52.719902 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:52.719946 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":58,"threads_started":1}
I20260924 21:14:52.858544 22245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:52.858372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.131:54451)
0924 21:14:52.858477 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:52.858506 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":55,"threads_started":1}
I20260924 21:14:53.633018 22246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:53.632779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.129:53101)
0924 21:14:53.632924 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:53.632973 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":64,"threads_started":1}
I20260924 21:14:53.720710 22247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:53.720484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.130:40425)
0924 21:14:53.720627 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:53.720670 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":64,"threads_started":1}
I20260924 21:14:53.859200 22248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:53.859006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.131:51373)
0924 21:14:53.859124 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:53.859162 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":61,"threads_started":1}
I20260924 21:14:54.304178 22249 raft_consensus.cc:482] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:14:54.304277 22249 raft_consensus.cc:504] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910de0a3bb54e69af8579664236d51b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46207 } } peers { permanent_uuid: "602408af328941bcbd447c7246078ff0" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43583 } } peers { permanent_uuid: "0993dfa60a2e4cef9e4246312d657aba" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40125 } }
I20260924 21:14:54.304442 22249 leader_election.cc:288] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f910de0a3bb54e69af8579664236d51b (127.6.210.131:46207), 0993dfa60a2e4cef9e4246312d657aba (127.6.210.130:40125)
W20260924 21:14:54.304913 21853 leader_election.cc:341] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f910de0a3bb54e69af8579664236d51b (127.6.210.131:46207): Not found: Tablet not found: 455681b4f2104a45991255bc23b1b7a1
W20260924 21:14:54.304996 21854 leader_election.cc:341] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0993dfa60a2e4cef9e4246312d657aba (127.6.210.130:40125): Not found: Tablet not found: 455681b4f2104a45991255bc23b1b7a1
I20260924 21:14:54.305063 21854 leader_election.cc:302] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 602408af328941bcbd447c7246078ff0; no voters: 0993dfa60a2e4cef9e4246312d657aba, f910de0a3bb54e69af8579664236d51b
I20260924 21:14:54.305166 22249 raft_consensus.cc:2726] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 21:14:54.633755 22250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:54.633513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.129:56497)
0924 21:14:54.633666 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:54.633709 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":65,"threads_started":1}
I20260924 21:14:54.721390 22251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:54.721170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.130:60425)
0924 21:14:54.721303 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:54.721336 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":62,"threads_started":1}
I20260924 21:14:54.859834 22252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:54.859605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.131:46159)
0924 21:14:54.859746 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:54.859792 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":60,"threads_started":1}
I20260924 21:14:55.634629 22253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:55.634367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.129:37285)
0924 21:14:55.634529 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:55.634575 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":76,"threads_started":1}
I20260924 21:14:55.722154 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:55.721893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.130:51451)
0924 21:14:55.722065 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:55.722109 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":75,"threads_started":1}
I20260924 21:14:55.860555 22255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:55.860331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.131:50269)
0924 21:14:55.860473 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:55.860512 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":76,"threads_started":1}
I20260924 21:14:56.635387 22256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:56.635141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.129:39889)
0924 21:14:56.635300 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:56.635341 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":66,"threads_started":1}
I20260924 21:14:56.722909 22257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:56.722620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.130:54003)
0924 21:14:56.722771 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:56.722816 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":72,"threads_started":1}
I20260924 21:14:56.861403 22258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:56.861134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.131:60365)
0924 21:14:56.861301 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:56.861351 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":61,"threads_started":1}
I20260924 21:14:57.131425 22259 raft_consensus.cc:482] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:14:57.131541 22259 raft_consensus.cc:504] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910de0a3bb54e69af8579664236d51b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46207 } } peers { permanent_uuid: "602408af328941bcbd447c7246078ff0" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43583 } } peers { permanent_uuid: "0993dfa60a2e4cef9e4246312d657aba" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40125 } }
I20260924 21:14:57.131742 22259 leader_election.cc:288] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f910de0a3bb54e69af8579664236d51b (127.6.210.131:46207), 0993dfa60a2e4cef9e4246312d657aba (127.6.210.130:40125)
W20260924 21:14:57.132270 21854 leader_election.cc:341] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0993dfa60a2e4cef9e4246312d657aba (127.6.210.130:40125): Not found: Tablet not found: 455681b4f2104a45991255bc23b1b7a1
W20260924 21:14:57.132359 21853 leader_election.cc:341] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f910de0a3bb54e69af8579664236d51b (127.6.210.131:46207): Not found: Tablet not found: 455681b4f2104a45991255bc23b1b7a1
I20260924 21:14:57.132406 21853 leader_election.cc:302] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 602408af328941bcbd447c7246078ff0; no voters: 0993dfa60a2e4cef9e4246312d657aba, f910de0a3bb54e69af8579664236d51b
I20260924 21:14:57.132507 22259 raft_consensus.cc:2726] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 21:14:57.636231 22260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:57.636033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.129:36009)
0924 21:14:57.636148 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:57.636187 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":77,"threads_started":1}
I20260924 21:14:57.723623 22261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:57.723399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.130:33387)
0924 21:14:57.723532 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:57.723574 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":60,"threads_started":1}
I20260924 21:14:57.862150 22262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:57.861922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.131:48293)
0924 21:14:57.862058 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:57.862106 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":73,"threads_started":1}
I20260924 21:14:58.637126 22263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:58.636814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.129:52965)
0924 21:14:58.637022 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:58.637075 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":66,"threads_started":1}
I20260924 21:14:58.724473 22264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:58.724204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.130:52805)
0924 21:14:58.724380 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:58.724424 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":79,"threads_started":1}
I20260924 21:14:58.862839 22265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:58.862639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.131:57485)
0924 21:14:58.862759 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:58.862793 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":60,"threads_started":1}
I20260924 21:14:59.554062 22266 raft_consensus.cc:482] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:14:59.554165 22266 raft_consensus.cc:504] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f910de0a3bb54e69af8579664236d51b" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 46207 } } peers { permanent_uuid: "602408af328941bcbd447c7246078ff0" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 43583 } } peers { permanent_uuid: "0993dfa60a2e4cef9e4246312d657aba" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40125 } }
I20260924 21:14:59.554412 22266 leader_election.cc:288] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f910de0a3bb54e69af8579664236d51b (127.6.210.131:46207), 0993dfa60a2e4cef9e4246312d657aba (127.6.210.130:40125)
W20260924 21:14:59.554869 21853 leader_election.cc:341] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f910de0a3bb54e69af8579664236d51b (127.6.210.131:46207): Not found: Tablet not found: 455681b4f2104a45991255bc23b1b7a1
W20260924 21:14:59.554970 21854 leader_election.cc:341] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0993dfa60a2e4cef9e4246312d657aba (127.6.210.130:40125): Not found: Tablet not found: 455681b4f2104a45991255bc23b1b7a1
I20260924 21:14:59.555055 21854 leader_election.cc:302] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 602408af328941bcbd447c7246078ff0; no voters: 0993dfa60a2e4cef9e4246312d657aba, f910de0a3bb54e69af8579664236d51b
I20260924 21:14:59.555163 22266 raft_consensus.cc:2726] T 455681b4f2104a45991255bc23b1b7a1 P 602408af328941bcbd447c7246078ff0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 21:14:59.637980 22267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:59.637731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.129:59355)
0924 21:14:59.637883 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:59.637931 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":80,"threads_started":1}
I20260924 21:14:59.725196 22268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:59.724968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.130:45591)
0924 21:14:59.725108 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:59.725150 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":73,"threads_started":1}
I20260924 21:14:59.863687 22269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:14:59.863463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.131:54461)
0924 21:14:59.863586 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0924 21:14:59.863631 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":67,"threads_started":1}
I20260924 21:15:00.638865 22270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:00.638597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.129:54635)
0924 21:15:00.638745 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:00.638789 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":68,"threads_started":1}
I20260924 21:15:00.725968 22271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:00.725732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.130:47599)
0924 21:15:00.725863 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:00.725919 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":64,"threads_started":1}
I20260924 21:15:00.864420 22272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:00.864188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.131:33847)
0924 21:15:00.864331 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:00.864373 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":67,"threads_started":1}
I20260924 21:15:01.639780 22273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:01.639495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.129:51879)
0924 21:15:01.639688 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:01.639733 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":73,"threads_started":1}
I20260924 21:15:01.726903 22274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:01.726528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.130:33843)
0924 21:15:01.726809 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:01.726857 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":69,"threads_started":1}
I20260924 21:15:01.865273 22275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:01.864987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.131:37895)
0924 21:15:01.865148 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:01.865224 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":78,"threads_started":1}
I20260924 21:15:02.640651 22276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:02.640384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.129:53985)
0924 21:15:02.640548 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:02.640597 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":90,"threads_started":1}
I20260924 21:15:02.727737 22277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:02.727501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.130:51859)
0924 21:15:02.727637 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:02.727669 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
I20260924 21:15:02.866115 22278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:02.865796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.131:52451)
0924 21:15:02.866028 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:02.866067 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":158,"threads_started":1}
I20260924 21:15:03.641325 22279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:03.641130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.129:33103)
0924 21:15:03.641246 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:03.641281 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":63,"threads_started":1}
I20260924 21:15:03.728479 22280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:03.728263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.130:35079)
0924 21:15:03.728389 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:03.728431 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":59,"threads_started":1}
I20260924 21:15:03.866983 22281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:03.866720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.131:60007)
0924 21:15:03.866860 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:03.866919 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":60,"threads_started":1}
I20260924 21:15:04.641984 22282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:04.641788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.129:43701)
0924 21:15:04.641911 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:04.641939 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":76,"threads_started":1}
I20260924 21:15:04.729223 22283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:04.728995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.130:48515)
0924 21:15:04.729136 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:04.729176 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":82,"threads_started":1}
I20260924 21:15:04.867740 22284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:04.867518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.131:41009)
0924 21:15:04.867654 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:04.867694 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":65,"threads_started":1}
I20260924 21:15:05.642896 22285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:05.642655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.129:43863)
0924 21:15:05.642798 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:05.642840 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":68,"threads_started":1}
I20260924 21:15:05.730059 22286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:05.729835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.130:54615)
0924 21:15:05.729971 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:05.730014 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":66,"threads_started":1}
I20260924 21:15:05.868168 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 21808
I20260924 21:15:05.869089 22287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:05.868382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:34097 (local address 127.6.210.131:48825)
0924 21:15:05.869003 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:05.869046 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:34097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":58,"threads_started":1}
I20260924 21:15:05.873142 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 21971
I20260924 21:15:05.877808 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 22103
2026-09-24T21:15:05Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18264 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:15:05 dist-test-slave-q276 krb5kdc[22290](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:15:05 dist-test-slave-q276 krb5kdc[22290](info): set up 2 sockets
Sep 24 21:15:05 dist-test-slave-q276 krb5kdc[22290](info): commencing operation
krb5kdc: starting...
W20260924 21:15:06.930857 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:15:07 dist-test-slave-q276 krb5kdc[22290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284507, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:15:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:15:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.210.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
I20260924 21:15:07.059096 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:36237
--webserver_interface=127.6.210.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:36793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:36237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:15:07.146445 22303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:15:07.146719 22303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:15:07.146759 22303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:15:07.148051 22303 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260924 21:15:07.148123 22303 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260924 21:15:07.148146 22303 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260924 21:15:07.148166 22303 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:15:07.148198 22303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:15:07.148231 22303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:15:07.148257 22303 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:15:07.148290 22303 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:15:07.149889 22303 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:36793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:36237
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:36237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:15:07.150177 22303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:15:07.150434 22303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:15:07.155827 22308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:15:07.155827 22311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:15:07.155908 22309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:15:07.155941 22303 server_base.cc:1034] running on GCE node
I20260924 21:15:07.156426 22303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:15:07.156709 22303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:15:07.157898 22303 hybrid_clock.cc:648] HybridClock initialized: now 1790284507157878 us; error 37 us; skew 500 ppm
Sep 24 21:15:07 dist-test-slave-q276 krb5kdc[22290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284507, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:15:07.160475 22303 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:15:07.160866 22303 webserver.cc:460] Webserver started at http://127.6.210.190:34323/ using document root <none> and password file <none>
I20260924 21:15:07.161069 22303 fs_manager.cc:359] Metadata directory not provided
I20260924 21:15:07.161111 22303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:15:07.161206 22303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:15:07.162240 22303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "cfb0af8fb6da4938802da89bde1d362b"
format_stamp: "Formatted at 2026-09-24 21:15:07 on dist-test-slave-q276"
server_key: "9a09c83fbe7f93b7f2ad83eb5b58731f"
server_key_iv: "84f522b76d313ced053fc82c1285b2cf"
server_key_version: "encryptionkey@0"
I20260924 21:15:07.162613 22303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "cfb0af8fb6da4938802da89bde1d362b"
format_stamp: "Formatted at 2026-09-24 21:15:07 on dist-test-slave-q276"
server_key: "9a09c83fbe7f93b7f2ad83eb5b58731f"
server_key_iv: "84f522b76d313ced053fc82c1285b2cf"
server_key_version: "encryptionkey@0"
I20260924 21:15:07.163945 22303 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:15:07.164654 22318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:15:07.164842 22303 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:15:07.164934 22303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "cfb0af8fb6da4938802da89bde1d362b"
format_stamp: "Formatted at 2026-09-24 21:15:07 on dist-test-slave-q276"
server_key: "9a09c83fbe7f93b7f2ad83eb5b58731f"
server_key_iv: "84f522b76d313ced053fc82c1285b2cf"
server_key_version: "encryptionkey@0"
I20260924 21:15:07.165060 22303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:15:07.173094 22303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:15:07.175271 22303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:15:07.175416 22303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:15:07.179280 22303 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:36237
I20260924 21:15:07.179312 22370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:36237 every 8 connection(s)
I20260924 21:15:07.179679 22303 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:15:07.180540 22371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:15:07.182883 22371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b: Bootstrap starting.
I20260924 21:15:07.183461 22371 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b: Neither blocks nor log segments found. Creating new log.
I20260924 21:15:07.183759 22371 log.cc:824] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b: Log is configured to *not* fsync() on all Append() calls
I20260924 21:15:07.184054 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 22303
I20260924 21:15:07.184182 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:15:07.184365 6986 external_mini_cluster.cc:1428] Setting key b023e2159455b99dd887a9c171725935
I20260924 21:15:07.184481 22371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b: No bootstrap required, opened a new log
I20260924 21:15:07.186316 22371 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb0af8fb6da4938802da89bde1d362b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36237 } }
I20260924 21:15:07.186462 22371 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:15:07.186549 22371 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfb0af8fb6da4938802da89bde1d362b, State: Initialized, Role: FOLLOWER
I20260924 21:15:07.186707 22371 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb0af8fb6da4938802da89bde1d362b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36237 } }
I20260924 21:15:07.186798 22371 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:15:07.186834 22371 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:15:07.186877 22371 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:15:07.187849 22371 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb0af8fb6da4938802da89bde1d362b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36237 } }
I20260924 21:15:07.187935 22371 leader_election.cc:302] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfb0af8fb6da4938802da89bde1d362b; no voters:
I20260924 21:15:07.188133 22371 leader_election.cc:288] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:15:07.188162 22376 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:15:07.188272 22376 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [term 1 LEADER]: Becoming Leader. State: Replica: cfb0af8fb6da4938802da89bde1d362b, State: Running, Role: LEADER
I20260924 21:15:07.188393 22371 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:15:07.188408 22376 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb0af8fb6da4938802da89bde1d362b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36237 } }
I20260924 21:15:07.188833 22376 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfb0af8fb6da4938802da89bde1d362b. Latest consensus state: current_term: 1 leader_uuid: "cfb0af8fb6da4938802da89bde1d362b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb0af8fb6da4938802da89bde1d362b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36237 } } }
I20260924 21:15:07.189087 22376 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [sys.catalog]: This master's current role is: LEADER
I20260924 21:15:07.189011 22378 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfb0af8fb6da4938802da89bde1d362b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb0af8fb6da4938802da89bde1d362b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36237 } } }
I20260924 21:15:07.189572 22378 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [sys.catalog]: This master's current role is: LEADER
I20260924 21:15:07.189710 22385 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:15:07.190353 22385 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:15:07.191880 22385 catalog_manager.cc:1347] Generated new cluster ID: 690c7d21ead34b058754485a3d03cf3f
I20260924 21:15:07.191936 22385 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 21:15:07 dist-test-slave-q276 krb5kdc[22290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284507, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:15:07.196414 22375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:07.184907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47038 (local address 127.6.210.190:36237)
0924 21:15:07.185072 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0924 21:15:07.185081 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:15:07.185114 (+ 33us) server_negotiation.cc:407] Connection header received
0924 21:15:07.186256 (+ 1142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:15:07.186316 (+ 60us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:15:07.186383 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:15:07.186537 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:15:07.187348 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:07.188491 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:07.189600 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:07.190178 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:07.190309 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:07.193240 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:15:07.193244 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:15:07.193244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:15:07.193254 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:15:07.194943 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:15:07.195300 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:15:07.195304 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:15:07.195304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:15:07.195407 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:15:07.195613 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:15:07.195613 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:15:07.195614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:15:07.195845 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:15:07.195991 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0924 21:15:07.196108 (+ 117us) server_negotiation.cc:299] Negotiation successful
0924 21:15:07.196256 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260924 21:15:07.207542 22385 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:15:07.208173 22385 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:15:07.210973 22385 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b: Generated new TSK 0
I20260924 21:15:07.211167 22385 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.210.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
I20260924 21:15:07.243605 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.129:0
--local_ip_for_outbound_sockets=127.6.210.129
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:36237
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:36793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:15:07.354413 22397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:15:07.354671 22397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:15:07.354702 22397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:15:07.355963 22397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:15:07.356005 22397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:15:07.356031 22397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.129
I20260924 21:15:07.357488 22397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:36793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:36237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.129
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:15:07.357741 22397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:15:07.357982 22397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:15:07.363153 22402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:15:07.363102 22403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:15:07.363343 22397 server_base.cc:1034] running on GCE node
W20260924 21:15:07.363380 22405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:15:07.363677 22397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:15:07.363966 22397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:15:07.365180 22397 hybrid_clock.cc:648] HybridClock initialized: now 1790284507365164 us; error 41 us; skew 500 ppm
Sep 24 21:15:07 dist-test-slave-q276 krb5kdc[22290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284507, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:15:07.368456 22397 init.cc:377] Logged in from keytab as kudu/127.6.210.129@KRBTEST.COM (short username kudu)
I20260924 21:15:07.369019 22397 webserver.cc:460] Webserver started at http://127.6.210.129:37825/ using document root <none> and password file <none>
I20260924 21:15:07.369282 22397 fs_manager.cc:359] Metadata directory not provided
I20260924 21:15:07.369352 22397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:15:07.369510 22397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:15:07.370745 22397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/data/instance:
uuid: "8e270e70ec6a4095ae7b556b25bebbe4"
format_stamp: "Formatted at 2026-09-24 21:15:07 on dist-test-slave-q276"
server_key: "9b091536e0c8998f3636ccaca6cb0250"
server_key_iv: "2d1accd6050ce22132e01fe04e10dde7"
server_key_version: "encryptionkey@0"
I20260924 21:15:07.371148 22397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance:
uuid: "8e270e70ec6a4095ae7b556b25bebbe4"
format_stamp: "Formatted at 2026-09-24 21:15:07 on dist-test-slave-q276"
server_key: "9b091536e0c8998f3636ccaca6cb0250"
server_key_iv: "2d1accd6050ce22132e01fe04e10dde7"
server_key_version: "encryptionkey@0"
I20260924 21:15:07.372586 22397 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:15:07.373533 22412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:15:07.373788 22397 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:15:07.373899 22397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/wal
uuid: "8e270e70ec6a4095ae7b556b25bebbe4"
format_stamp: "Formatted at 2026-09-24 21:15:07 on dist-test-slave-q276"
server_key: "9b091536e0c8998f3636ccaca6cb0250"
server_key_iv: "2d1accd6050ce22132e01fe04e10dde7"
server_key_version: "encryptionkey@0"
I20260924 21:15:07.373978 22397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:15:07.391500 22397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:15:07.393599 22397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:15:07.393771 22397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:15:07.394016 22397 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:15:07.394421 22397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:15:07.394480 22397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:15:07.394522 22397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:15:07.394549 22397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:15:07.400702 22397 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.129:32817
I20260924 21:15:07.400794 22525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.129:32817 every 8 connection(s)
I20260924 21:15:07.401120 22397 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
Sep 24 21:15:07 dist-test-slave-q276 krb5kdc[22290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284507, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:15:07.409611 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 22397
I20260924 21:15:07.409724 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance
I20260924 21:15:07.409952 6986 external_mini_cluster.cc:1428] Setting key b1233f1ccae2b3a51c1ce6868ce1287a
I20260924 21:15:07.411594 22375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:07.401881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:55267 (local address 127.6.210.190:36237)
0924 21:15:07.401958 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0924 21:15:07.401960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:15:07.402251 (+ 291us) server_negotiation.cc:407] Connection header received
0924 21:15:07.402879 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:15:07.402880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:15:07.402894 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:15:07.402967 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:15:07.404040 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:07.404585 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:07.405623 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:07.405845 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:07.405904 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:07.408374 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:15:07.408376 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:15:07.408377 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:15:07.408385 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:15:07.409988 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:15:07.410513 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:15:07.410514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:15:07.410514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:15:07.410584 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:15:07.410879 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:15:07.410880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:15:07.410880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:15:07.410998 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:15:07.411058 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 21:15:07.411379 (+ 321us) server_negotiation.cc:299] Negotiation successful
0924 21:15:07.411476 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260924 21:15:07.411753 22528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:07.401856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.129:55267)
0924 21:15:07.402118 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:07.402160 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 21:15:07.402782 (+ 622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:15:07.403043 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:15:07.403047 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:15:07.403708 (+ 661us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:15:07.403924 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:07.403930 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:07.404705 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:07.404706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:07.405537 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:07.405539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:07.405921 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:07.405922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:07.406068 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:07.406599 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:15:07.406607 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:15:07.408274 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:15:07.410091 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:15:07.410099 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:15:07.410102 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:15:07.410440 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:15:07.410682 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:15:07.410683 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:15:07.410683 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:15:07.410789 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:15:07.411104 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:15:07.411114 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:15:07.411238 (+ 124us) client_negotiation.cc:769] Sending connection context
0924 21:15:07.411340 (+ 102us) client_negotiation.cc:240] Negotiation successful
0924 21:15:07.411512 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":155,"threads_started":1}
I20260924 21:15:07.412196 22526 heartbeater.cc:347] Connected to a master server at 127.6.210.190:36237
I20260924 21:15:07.412315 22526 heartbeater.cc:464] Registering TS with master...
I20260924 21:15:07.412511 22526 heartbeater.cc:511] Master 127.6.210.190:36237 requested a full tablet report, sending...
I20260924 21:15:07.413075 22335 ts_manager.cc:194] Registered new tserver with Master: 8e270e70ec6a4095ae7b556b25bebbe4 (127.6.210.129:32817)
I20260924 21:15:07.413779 22335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.129@KRBTEST.COM'} at 127.6.210.129:55267
WARNING: no policy specified for kudu/127.6.210.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
I20260924 21:15:07.445631 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.130:0
--local_ip_for_outbound_sockets=127.6.210.130
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:36237
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:36793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:15:07.530364 22531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:15:07.530577 22531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:15:07.530601 22531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:15:07.531852 22531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:15:07.531900 22531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:15:07.531929 22531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.130
I20260924 21:15:07.533334 22531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:36793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:36237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.130
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:15:07.533524 22531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:15:07.533720 22531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:15:07.538730 22537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:15:07.538726 22536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:15:07.538790 22539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:15:07.539095 22531 server_base.cc:1034] running on GCE node
I20260924 21:15:07.539335 22531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:15:07.539637 22531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:15:07.540815 22531 hybrid_clock.cc:648] HybridClock initialized: now 1790284507540789 us; error 35 us; skew 500 ppm
Sep 24 21:15:07 dist-test-slave-q276 krb5kdc[22290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284507, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:15:07.547504 22531 init.cc:377] Logged in from keytab as kudu/127.6.210.130@KRBTEST.COM (short username kudu)
I20260924 21:15:07.548046 22531 webserver.cc:460] Webserver started at http://127.6.210.130:42323/ using document root <none> and password file <none>
I20260924 21:15:07.548307 22531 fs_manager.cc:359] Metadata directory not provided
I20260924 21:15:07.548382 22531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:15:07.548547 22531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:15:07.549819 22531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/data/instance:
uuid: "1eeca863d1c14311add655d8b95dce4b"
format_stamp: "Formatted at 2026-09-24 21:15:07 on dist-test-slave-q276"
server_key: "f2f4205afb85a38ff8b17056d9e1a2f7"
server_key_iv: "0bc83b69a687accadb72f95e15172476"
server_key_version: "encryptionkey@0"
I20260924 21:15:07.550218 22531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance:
uuid: "1eeca863d1c14311add655d8b95dce4b"
format_stamp: "Formatted at 2026-09-24 21:15:07 on dist-test-slave-q276"
server_key: "f2f4205afb85a38ff8b17056d9e1a2f7"
server_key_iv: "0bc83b69a687accadb72f95e15172476"
server_key_version: "encryptionkey@0"
I20260924 21:15:07.551899 22531 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:15:07.552755 22546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:15:07.552955 22531 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:15:07.553053 22531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/wal
uuid: "1eeca863d1c14311add655d8b95dce4b"
format_stamp: "Formatted at 2026-09-24 21:15:07 on dist-test-slave-q276"
server_key: "f2f4205afb85a38ff8b17056d9e1a2f7"
server_key_iv: "0bc83b69a687accadb72f95e15172476"
server_key_version: "encryptionkey@0"
I20260924 21:15:07.553174 22531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:15:07.571444 22531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:15:07.573886 22531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:15:07.574087 22531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:15:07.574429 22531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:15:07.574783 22531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:15:07.574846 22531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:15:07.574892 22531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:15:07.574923 22531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:15:07.581460 22531 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.130:42013
I20260924 21:15:07.581491 22659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.130:42013 every 8 connection(s)
I20260924 21:15:07.581856 22531 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
Sep 24 21:15:07 dist-test-slave-q276 krb5kdc[22290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284507, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:15:07.591382 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 22531
I20260924 21:15:07.591472 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance
I20260924 21:15:07.591697 6986 external_mini_cluster.cc:1428] Setting key d8de0a70d1af89a5d29b5a7cf3cb88dd
I20260924 21:15:07.591760 22375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:07.582658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:38369 (local address 127.6.210.190:36237)
0924 21:15:07.582731 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0924 21:15:07.582733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:15:07.582937 (+ 204us) server_negotiation.cc:407] Connection header received
0924 21:15:07.583492 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:15:07.583493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:15:07.583504 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:15:07.583565 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:15:07.584604 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:07.585141 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:07.586173 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:07.586462 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:07.586531 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:07.588678 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:15:07.588680 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:15:07.588680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:15:07.588687 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:15:07.590152 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:15:07.590606 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:15:07.590607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:15:07.590608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:15:07.590667 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:15:07.590914 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:15:07.590914 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:15:07.590915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:15:07.591088 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:15:07.591150 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 21:15:07.591565 (+ 415us) server_negotiation.cc:299] Negotiation successful
0924 21:15:07.591668 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 21:15:07.591935 22662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:07.582645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.130:38369)
0924 21:15:07.582825 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:07.582864 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 21:15:07.583383 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:15:07.583643 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:15:07.583646 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:15:07.584274 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:15:07.584478 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:07.584484 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:07.585250 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:07.585251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:07.586077 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:07.586079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:07.586548 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:07.586549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:07.586665 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:07.587182 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:15:07.587190 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:15:07.588587 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:15:07.590228 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:15:07.590234 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:15:07.590236 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:15:07.590503 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:15:07.590732 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:15:07.590732 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:15:07.590733 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:15:07.590846 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:15:07.591158 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:15:07.591206 (+ 48us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:15:07.591355 (+ 149us) client_negotiation.cc:769] Sending connection context
0924 21:15:07.591495 (+ 140us) client_negotiation.cc:240] Negotiation successful
0924 21:15:07.591686 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":89,"threads_started":1}
I20260924 21:15:07.592813 22660 heartbeater.cc:347] Connected to a master server at 127.6.210.190:36237
I20260924 21:15:07.592919 22660 heartbeater.cc:464] Registering TS with master...
I20260924 21:15:07.593112 22660 heartbeater.cc:511] Master 127.6.210.190:36237 requested a full tablet report, sending...
I20260924 21:15:07.593508 22335 ts_manager.cc:194] Registered new tserver with Master: 1eeca863d1c14311add655d8b95dce4b (127.6.210.130:42013)
I20260924 21:15:07.593884 22335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.130@KRBTEST.COM'} at 127.6.210.130:38369
WARNING: no policy specified for kudu/127.6.210.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
I20260924 21:15:07.618542 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.131:0
--local_ip_for_outbound_sockets=127.6.210.131
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:36237
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:36793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:15:07.701507 22665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:15:07.701730 22665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:15:07.701761 22665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:15:07.702947 22665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:15:07.703020 22665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:15:07.703068 22665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.131
I20260924 21:15:07.704530 22665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:36793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:36237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.131
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:15:07.704785 22665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:15:07.705015 22665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:15:07.709924 22671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:15:07.709944 22670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:15:07.709924 22673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:15:07.710244 22665 server_base.cc:1034] running on GCE node
I20260924 21:15:07.710623 22665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:15:07.710904 22665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:15:07.712052 22665 hybrid_clock.cc:648] HybridClock initialized: now 1790284507712025 us; error 43 us; skew 500 ppm
Sep 24 21:15:07 dist-test-slave-q276 krb5kdc[22290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284507, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:15:07.714704 22665 init.cc:377] Logged in from keytab as kudu/127.6.210.131@KRBTEST.COM (short username kudu)
I20260924 21:15:07.715173 22665 webserver.cc:460] Webserver started at http://127.6.210.131:42977/ using document root <none> and password file <none>
I20260924 21:15:07.715400 22665 fs_manager.cc:359] Metadata directory not provided
I20260924 21:15:07.715444 22665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:15:07.715600 22665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:15:07.716637 22665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/data/instance:
uuid: "4100450f8fde416ba78f1bdb33a22f16"
format_stamp: "Formatted at 2026-09-24 21:15:07 on dist-test-slave-q276"
server_key: "90f1a61a32a75db61de042f759f25e54"
server_key_iv: "68a21c0e11bfe1df6aaf82846db47a26"
server_key_version: "encryptionkey@0"
I20260924 21:15:07.717000 22665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance:
uuid: "4100450f8fde416ba78f1bdb33a22f16"
format_stamp: "Formatted at 2026-09-24 21:15:07 on dist-test-slave-q276"
server_key: "90f1a61a32a75db61de042f759f25e54"
server_key_iv: "68a21c0e11bfe1df6aaf82846db47a26"
server_key_version: "encryptionkey@0"
I20260924 21:15:07.718410 22665 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 21:15:07.719283 22680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:15:07.719523 22665 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:15:07.719635 22665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/wal
uuid: "4100450f8fde416ba78f1bdb33a22f16"
format_stamp: "Formatted at 2026-09-24 21:15:07 on dist-test-slave-q276"
server_key: "90f1a61a32a75db61de042f759f25e54"
server_key_iv: "68a21c0e11bfe1df6aaf82846db47a26"
server_key_version: "encryptionkey@0"
I20260924 21:15:07.719738 22665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:15:07.735967 22665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:15:07.738091 22665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:15:07.738306 22665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:15:07.738565 22665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:15:07.738883 22665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:15:07.738935 22665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:15:07.738981 22665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:15:07.739006 22665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:15:07.745126 22665 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.131:38743
I20260924 21:15:07.745200 22793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.131:38743 every 8 connection(s)
I20260924 21:15:07.745558 22665 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
Sep 24 21:15:07 dist-test-slave-q276 krb5kdc[22290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284507, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:15:07.753800 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 22665
I20260924 21:15:07.753903 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance
I20260924 21:15:07.754135 6986 external_mini_cluster.cc:1428] Setting key badb8c30188d779c37ca68dd73d8747e
I20260924 21:15:07.755178 22375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:07.746323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:46927 (local address 127.6.210.190:36237)
0924 21:15:07.746370 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 21:15:07.746372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:15:07.746586 (+ 214us) server_negotiation.cc:407] Connection header received
0924 21:15:07.747129 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:15:07.747129 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:15:07.747140 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:15:07.747192 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:15:07.748170 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:07.748693 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:07.749776 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:07.749962 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:07.750019 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:07.752251 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:15:07.752252 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:15:07.752252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:15:07.752259 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:15:07.753502 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:15:07.753902 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:15:07.753904 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:15:07.753904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:15:07.753992 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:15:07.754439 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:15:07.754440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:15:07.754441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:15:07.754616 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:15:07.754688 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 21:15:07.755029 (+ 341us) server_negotiation.cc:299] Negotiation successful
0924 21:15:07.755117 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 21:15:07.755400 22796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:07.746319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.131:46927)
0924 21:15:07.746465 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:07.746510 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 21:15:07.747005 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:15:07.747251 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:15:07.747255 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:15:07.747877 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:15:07.748091 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:07.748096 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:07.748778 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:07.748779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:07.749688 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:07.749691 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:07.750038 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:07.750038 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:07.750140 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:07.750692 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:15:07.750700 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:15:07.752157 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:15:07.753573 (+ 1416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:15:07.753579 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:15:07.753581 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:15:07.753811 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:15:07.754077 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:15:07.754078 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:15:07.754079 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:15:07.754218 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:15:07.754719 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:15:07.754725 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:15:07.754862 (+ 137us) client_negotiation.cc:769] Sending connection context
0924 21:15:07.754986 (+ 124us) client_negotiation.cc:240] Negotiation successful
0924 21:15:07.755139 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":82,"threads_started":1}
I20260924 21:15:07.755863 22794 heartbeater.cc:347] Connected to a master server at 127.6.210.190:36237
I20260924 21:15:07.755954 22794 heartbeater.cc:464] Registering TS with master...
I20260924 21:15:07.756108 22794 heartbeater.cc:511] Master 127.6.210.190:36237 requested a full tablet report, sending...
I20260924 21:15:07.756464 22335 ts_manager.cc:194] Registered new tserver with Master: 4100450f8fde416ba78f1bdb33a22f16 (127.6.210.131:38743)
I20260924 21:15:07.756881 22335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.131@KRBTEST.COM'} at 127.6.210.131:46927
I20260924 21:15:07.766144 6986 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:15:07.774237 22375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:07.767578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47040 (local address 127.6.210.190:36237)
0924 21:15:07.767657 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0924 21:15:07.767659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:15:07.767720 (+ 61us) server_negotiation.cc:407] Connection header received
0924 21:15:07.767802 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:15:07.767803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:15:07.767813 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:15:07.767897 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:15:07.768566 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:07.769083 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:07.769814 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:07.770004 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:07.770073 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:07.771323 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:15:07.771325 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:15:07.771325 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:15:07.771332 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:15:07.772841 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:15:07.773314 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:15:07.773316 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:15:07.773316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:15:07.773378 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:15:07.773589 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:15:07.773590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:15:07.773591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:15:07.773755 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:15:07.773868 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0924 21:15:07.774042 (+ 174us) server_negotiation.cc:299] Negotiation successful
0924 21:15:07.774125 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260924 21:15:07.775955 22335 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47040:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:15:07.777724 22335 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:15:07.784217 22806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:07.780412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59452 (local address 127.6.210.131:38743)
0924 21:15:07.780594 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0924 21:15:07.780604 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:15:07.780923 (+ 319us) server_negotiation.cc:407] Connection header received
0924 21:15:07.780976 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:15:07.780978 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:15:07.780998 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:15:07.781078 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:15:07.781282 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:07.782055 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:07.783270 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:07.783771 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:07.783858 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:07.783920 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 21:15:07.784084 (+ 164us) server_negotiation.cc:299] Negotiation successful
0924 21:15:07.784129 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":92,"threads_started":1}
I20260924 21:15:07.784205 22808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:07.780322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:38743 (local address 127.0.0.1:59452)
0924 21:15:07.780695 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:07.780736 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 21:15:07.780819 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:15:07.781104 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:15:07.781108 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:15:07.781111 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:15:07.781184 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:07.781186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:07.782145 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:07.782146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:07.783186 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:07.783188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:07.783859 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:07.783860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:07.783947 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:07.783967 (+ 20us) client_negotiation.cc:769] Sending connection context
0924 21:15:07.784067 (+ 100us) client_negotiation.cc:240] Negotiation successful
0924 21:15:07.784126 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":54,"threads_started":1}
I20260924 21:15:07.785135 22728 tablet_service.cc:1467] Processing CreateTablet for tablet 1c3561677beb4ae09740e00943f7d295 (DEFAULT_TABLE table=test-table [id=3b27794a09734938b654feb62b7e0238]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:15:07.785467 22805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:07.780382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:42013 (local address 127.0.0.1:54080)
0924 21:15:07.780706 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:07.780909 (+ 203us) client_negotiation.cc:174] Beginning negotiation
0924 21:15:07.781019 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:15:07.781371 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:15:07.781372 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:15:07.781374 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:15:07.781416 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:07.781418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:07.782739 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:07.782740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:07.784010 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:07.784013 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:07.785151 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:07.785153 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:07.785231 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:07.785236 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 21:15:07.785299 (+ 63us) client_negotiation.cc:240] Negotiation successful
0924 21:15:07.785352 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":57,"thread_start_us":136,"threads_started":1}
I20260924 21:15:07.785521 22728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c3561677beb4ae09740e00943f7d295. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:15:07.785818 22809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:07.780696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54080 (local address 127.6.210.130:42013)
0924 21:15:07.781197 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0924 21:15:07.781200 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:15:07.781228 (+ 28us) server_negotiation.cc:407] Connection header received
0924 21:15:07.781268 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:15:07.781270 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:15:07.781285 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:15:07.781375 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:15:07.781477 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:07.782617 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:07.784320 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:07.785072 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:07.785145 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:07.785201 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 21:15:07.785676 (+ 475us) server_negotiation.cc:299] Negotiation successful
0924 21:15:07.785728 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":96,"threads_started":1}
I20260924 21:15:07.786528 22807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:07.780383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:32817 (local address 127.0.0.1:47838)
0924 21:15:07.780760 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:07.780769 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:15:07.780819 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:15:07.781256 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:15:07.781257 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:15:07.781260 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:15:07.781314 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:07.781316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:07.782744 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:07.782744 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:07.784010 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:07.784013 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:07.786218 (+ 2205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:07.786219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:07.786321 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:07.786327 (+ 6us) client_negotiation.cc:769] Sending connection context
0924 21:15:07.786391 (+ 64us) client_negotiation.cc:240] Negotiation successful
0924 21:15:07.786436 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"mutex_wait_us":69,"thread_start_us":123,"threads_started":1}
I20260924 21:15:07.786556 22810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:07.780890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47838 (local address 127.6.210.129:32817)
0924 21:15:07.781081 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0924 21:15:07.781086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:15:07.781096 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:15:07.781136 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:15:07.781139 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:15:07.781153 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:15:07.781249 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:15:07.781412 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:07.782611 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:07.785386 (+ 2775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:07.786123 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:07.786195 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:07.786248 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 21:15:07.786417 (+ 169us) server_negotiation.cc:299] Negotiation successful
0924 21:15:07.786464 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":79,"threads_started":1}
I20260924 21:15:07.786543 22594 tablet_service.cc:1467] Processing CreateTablet for tablet 1c3561677beb4ae09740e00943f7d295 (DEFAULT_TABLE table=test-table [id=3b27794a09734938b654feb62b7e0238]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:15:07.786900 22594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c3561677beb4ae09740e00943f7d295. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:15:07.787560 22460 tablet_service.cc:1467] Processing CreateTablet for tablet 1c3561677beb4ae09740e00943f7d295 (DEFAULT_TABLE table=test-table [id=3b27794a09734938b654feb62b7e0238]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:15:07.787923 22460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c3561677beb4ae09740e00943f7d295. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:15:07.788784 22811 tablet_bootstrap.cc:492] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16: Bootstrap starting.
I20260924 21:15:07.789425 22811 tablet_bootstrap.cc:654] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16: Neither blocks nor log segments found. Creating new log.
I20260924 21:15:07.789882 22811 log.cc:824] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16: Log is configured to *not* fsync() on all Append() calls
I20260924 21:15:07.790690 22811 tablet_bootstrap.cc:492] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16: No bootstrap required, opened a new log
I20260924 21:15:07.790771 22811 ts_tablet_manager.cc:1397] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:15:07.790845 22813 tablet_bootstrap.cc:492] T 1c3561677beb4ae09740e00943f7d295 P 1eeca863d1c14311add655d8b95dce4b: Bootstrap starting.
I20260924 21:15:07.791566 22813 tablet_bootstrap.cc:654] T 1c3561677beb4ae09740e00943f7d295 P 1eeca863d1c14311add655d8b95dce4b: Neither blocks nor log segments found. Creating new log.
I20260924 21:15:07.791736 22814 tablet_bootstrap.cc:492] T 1c3561677beb4ae09740e00943f7d295 P 8e270e70ec6a4095ae7b556b25bebbe4: Bootstrap starting.
I20260924 21:15:07.792192 22813 log.cc:824] T 1c3561677beb4ae09740e00943f7d295 P 1eeca863d1c14311add655d8b95dce4b: Log is configured to *not* fsync() on all Append() calls
I20260924 21:15:07.792472 22814 tablet_bootstrap.cc:654] T 1c3561677beb4ae09740e00943f7d295 P 8e270e70ec6a4095ae7b556b25bebbe4: Neither blocks nor log segments found. Creating new log.
I20260924 21:15:07.792483 22811 raft_consensus.cc:348] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4100450f8fde416ba78f1bdb33a22f16" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 38743 } } peers { permanent_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 32817 } } peers { permanent_uuid: "1eeca863d1c14311add655d8b95dce4b" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 42013 } }
I20260924 21:15:07.792694 22811 raft_consensus.cc:374] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:15:07.792738 22811 raft_consensus.cc:729] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4100450f8fde416ba78f1bdb33a22f16, State: Initialized, Role: FOLLOWER
I20260924 21:15:07.792858 22811 consensus_queue.cc:260] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4100450f8fde416ba78f1bdb33a22f16" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 38743 } } peers { permanent_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 32817 } } peers { permanent_uuid: "1eeca863d1c14311add655d8b95dce4b" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 42013 } }
I20260924 21:15:07.793138 22794 heartbeater.cc:503] Master 127.6.210.190:36237 was elected leader, sending a full tablet report...
I20260924 21:15:07.793144 22814 log.cc:824] T 1c3561677beb4ae09740e00943f7d295 P 8e270e70ec6a4095ae7b556b25bebbe4: Log is configured to *not* fsync() on all Append() calls
I20260924 21:15:07.793138 22811 ts_tablet_manager.cc:1428] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 21:15:07.793396 22813 tablet_bootstrap.cc:492] T 1c3561677beb4ae09740e00943f7d295 P 1eeca863d1c14311add655d8b95dce4b: No bootstrap required, opened a new log
I20260924 21:15:07.793469 22813 ts_tablet_manager.cc:1397] T 1c3561677beb4ae09740e00943f7d295 P 1eeca863d1c14311add655d8b95dce4b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260924 21:15:07.793910 22814 tablet_bootstrap.cc:492] T 1c3561677beb4ae09740e00943f7d295 P 8e270e70ec6a4095ae7b556b25bebbe4: No bootstrap required, opened a new log
I20260924 21:15:07.793988 22814 ts_tablet_manager.cc:1397] T 1c3561677beb4ae09740e00943f7d295 P 8e270e70ec6a4095ae7b556b25bebbe4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:15:07.795670 22813 raft_consensus.cc:348] T 1c3561677beb4ae09740e00943f7d295 P 1eeca863d1c14311add655d8b95dce4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4100450f8fde416ba78f1bdb33a22f16" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 38743 } } peers { permanent_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 32817 } } peers { permanent_uuid: "1eeca863d1c14311add655d8b95dce4b" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 42013 } }
I20260924 21:15:07.795864 22813 raft_consensus.cc:374] T 1c3561677beb4ae09740e00943f7d295 P 1eeca863d1c14311add655d8b95dce4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:15:07.795904 22813 raft_consensus.cc:729] T 1c3561677beb4ae09740e00943f7d295 P 1eeca863d1c14311add655d8b95dce4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1eeca863d1c14311add655d8b95dce4b, State: Initialized, Role: FOLLOWER
I20260924 21:15:07.796041 22813 consensus_queue.cc:260] T 1c3561677beb4ae09740e00943f7d295 P 1eeca863d1c14311add655d8b95dce4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4100450f8fde416ba78f1bdb33a22f16" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 38743 } } peers { permanent_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 32817 } } peers { permanent_uuid: "1eeca863d1c14311add655d8b95dce4b" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 42013 } }
I20260924 21:15:07.796128 22814 raft_consensus.cc:348] T 1c3561677beb4ae09740e00943f7d295 P 8e270e70ec6a4095ae7b556b25bebbe4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4100450f8fde416ba78f1bdb33a22f16" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 38743 } } peers { permanent_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 32817 } } peers { permanent_uuid: "1eeca863d1c14311add655d8b95dce4b" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 42013 } }
I20260924 21:15:07.796310 22814 raft_consensus.cc:374] T 1c3561677beb4ae09740e00943f7d295 P 8e270e70ec6a4095ae7b556b25bebbe4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:15:07.796358 22813 ts_tablet_manager.cc:1428] T 1c3561677beb4ae09740e00943f7d295 P 1eeca863d1c14311add655d8b95dce4b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:15:07.796358 22814 raft_consensus.cc:729] T 1c3561677beb4ae09740e00943f7d295 P 8e270e70ec6a4095ae7b556b25bebbe4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e270e70ec6a4095ae7b556b25bebbe4, State: Initialized, Role: FOLLOWER
I20260924 21:15:07.796379 22660 heartbeater.cc:503] Master 127.6.210.190:36237 was elected leader, sending a full tablet report...
I20260924 21:15:07.796479 22814 consensus_queue.cc:260] T 1c3561677beb4ae09740e00943f7d295 P 8e270e70ec6a4095ae7b556b25bebbe4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4100450f8fde416ba78f1bdb33a22f16" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 38743 } } peers { permanent_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 32817 } } peers { permanent_uuid: "1eeca863d1c14311add655d8b95dce4b" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 42013 } }
I20260924 21:15:07.796815 22814 ts_tablet_manager.cc:1428] T 1c3561677beb4ae09740e00943f7d295 P 8e270e70ec6a4095ae7b556b25bebbe4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:15:07.796919 22526 heartbeater.cc:503] Master 127.6.210.190:36237 was elected leader, sending a full tablet report...
W20260924 21:15:07.832436 22661 tablet.cc:2369] T 1c3561677beb4ae09740e00943f7d295 P 1eeca863d1c14311add655d8b95dce4b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 21:15:07.901901 22527 tablet.cc:2369] T 1c3561677beb4ae09740e00943f7d295 P 8e270e70ec6a4095ae7b556b25bebbe4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 21:15:07.996151 22795 tablet.cc:2369] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:15:08.044055 22816 raft_consensus.cc:482] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:15:08.044215 22816 raft_consensus.cc:504] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4100450f8fde416ba78f1bdb33a22f16" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 38743 } } peers { permanent_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 32817 } } peers { permanent_uuid: "1eeca863d1c14311add655d8b95dce4b" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 42013 } }
I20260924 21:15:08.044685 22816 leader_election.cc:288] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e270e70ec6a4095ae7b556b25bebbe4 (127.6.210.129:32817), 1eeca863d1c14311add655d8b95dce4b (127.6.210.130:42013)
I20260924 21:15:08.047793 22809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:08.044964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:57737 (local address 127.6.210.130:42013)
0924 21:15:08.045015 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 21:15:08.045017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:15:08.045029 (+ 12us) server_negotiation.cc:407] Connection header received
0924 21:15:08.045151 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:15:08.045152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:15:08.045163 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:15:08.045238 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:15:08.045403 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:08.045942 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:08.046992 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:08.047402 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:08.047497 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:08.047507 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 21:15:08.047670 (+ 163us) server_negotiation.cc:299] Negotiation successful
0924 21:15:08.047712 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 21:15:08.047806 22810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:08.045072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:37895 (local address 127.6.210.129:32817)
0924 21:15:08.045133 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 21:15:08.045135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:15:08.045362 (+ 227us) server_negotiation.cc:407] Connection header received
0924 21:15:08.045444 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:15:08.045445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:15:08.045455 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:15:08.045520 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:15:08.045699 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:08.046141 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:08.047108 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:08.047464 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:08.047541 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:08.047549 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:15:08.047700 (+ 151us) server_negotiation.cc:299] Negotiation successful
0924 21:15:08.047733 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 21:15:08.047832 22796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:08.044913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:42013 (local address 127.6.210.131:57737)
0924 21:15:08.044976 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:08.044988 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:15:08.045040 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:15:08.045241 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:15:08.045242 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:15:08.045243 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:15:08.045326 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:08.045328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:08.046033 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:08.046034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:08.046865 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:08.046867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:08.047507 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:08.047508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:08.047590 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:08.047593 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:15:08.047664 (+ 71us) client_negotiation.cc:240] Negotiation successful
0924 21:15:08.047708 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260924 21:15:08.047834 22820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:08.044986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:32817 (local address 127.6.210.131:37895)
0924 21:15:08.045284 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:08.045294 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:15:08.045375 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:15:08.045522 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:15:08.045523 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:15:08.045524 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:15:08.045601 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:08.045602 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:08.046242 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:08.046243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:08.047020 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:08.047022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:08.047554 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:08.047555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:08.047629 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:08.047632 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:15:08.047693 (+ 61us) client_negotiation.cc:240] Negotiation successful
0924 21:15:08.047733 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":230,"threads_started":1}
I20260924 21:15:08.048225 22614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c3561677beb4ae09740e00943f7d295" candidate_uuid: "4100450f8fde416ba78f1bdb33a22f16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1eeca863d1c14311add655d8b95dce4b" is_pre_election: true
I20260924 21:15:08.048261 22478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c3561677beb4ae09740e00943f7d295" candidate_uuid: "4100450f8fde416ba78f1bdb33a22f16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" is_pre_election: true
I20260924 21:15:08.048405 22478 raft_consensus.cc:2454] T 1c3561677beb4ae09740e00943f7d295 P 8e270e70ec6a4095ae7b556b25bebbe4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4100450f8fde416ba78f1bdb33a22f16 in term 0.
I20260924 21:15:08.048394 22614 raft_consensus.cc:2454] T 1c3561677beb4ae09740e00943f7d295 P 1eeca863d1c14311add655d8b95dce4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4100450f8fde416ba78f1bdb33a22f16 in term 0.
I20260924 21:15:08.048645 22682 leader_election.cc:302] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1eeca863d1c14311add655d8b95dce4b, 4100450f8fde416ba78f1bdb33a22f16; no voters:
I20260924 21:15:08.048790 22816 raft_consensus.cc:2781] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:15:08.048856 22816 raft_consensus.cc:482] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:15:08.048875 22816 raft_consensus.cc:3037] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:15:08.049651 22816 raft_consensus.cc:504] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4100450f8fde416ba78f1bdb33a22f16" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 38743 } } peers { permanent_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 32817 } } peers { permanent_uuid: "1eeca863d1c14311add655d8b95dce4b" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 42013 } }
I20260924 21:15:08.049811 22816 leader_election.cc:288] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16 [CANDIDATE]: Term 1 election: Requested vote from peers 8e270e70ec6a4095ae7b556b25bebbe4 (127.6.210.129:32817), 1eeca863d1c14311add655d8b95dce4b (127.6.210.130:42013)
I20260924 21:15:08.050011 22614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c3561677beb4ae09740e00943f7d295" candidate_uuid: "4100450f8fde416ba78f1bdb33a22f16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1eeca863d1c14311add655d8b95dce4b"
I20260924 21:15:08.050029 22478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c3561677beb4ae09740e00943f7d295" candidate_uuid: "4100450f8fde416ba78f1bdb33a22f16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e270e70ec6a4095ae7b556b25bebbe4"
I20260924 21:15:08.050103 22614 raft_consensus.cc:3037] T 1c3561677beb4ae09740e00943f7d295 P 1eeca863d1c14311add655d8b95dce4b [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:15:08.050103 22478 raft_consensus.cc:3037] T 1c3561677beb4ae09740e00943f7d295 P 8e270e70ec6a4095ae7b556b25bebbe4 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:15:08.051177 22614 raft_consensus.cc:2454] T 1c3561677beb4ae09740e00943f7d295 P 1eeca863d1c14311add655d8b95dce4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4100450f8fde416ba78f1bdb33a22f16 in term 1.
I20260924 21:15:08.051183 22478 raft_consensus.cc:2454] T 1c3561677beb4ae09740e00943f7d295 P 8e270e70ec6a4095ae7b556b25bebbe4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4100450f8fde416ba78f1bdb33a22f16 in term 1.
I20260924 21:15:08.051389 22683 leader_election.cc:302] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4100450f8fde416ba78f1bdb33a22f16, 8e270e70ec6a4095ae7b556b25bebbe4; no voters:
I20260924 21:15:08.051493 22816 raft_consensus.cc:2781] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:15:08.051635 22816 raft_consensus.cc:686] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16 [term 1 LEADER]: Becoming Leader. State: Replica: 4100450f8fde416ba78f1bdb33a22f16, State: Running, Role: LEADER
I20260924 21:15:08.051739 22816 consensus_queue.cc:237] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4100450f8fde416ba78f1bdb33a22f16" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 38743 } } peers { permanent_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 32817 } } peers { permanent_uuid: "1eeca863d1c14311add655d8b95dce4b" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 42013 } }
I20260924 21:15:08.052660 22334 catalog_manager.cc:5495] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4100450f8fde416ba78f1bdb33a22f16 (127.6.210.131). New cstate: current_term: 1 leader_uuid: "4100450f8fde416ba78f1bdb33a22f16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4100450f8fde416ba78f1bdb33a22f16" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 38743 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 32817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1eeca863d1c14311add655d8b95dce4b" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 42013 } health_report { overall_health: UNKNOWN } } }
I20260924 21:15:08.067366 22806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:08.064758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59460 (local address 127.6.210.131:38743)
0924 21:15:08.064820 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 21:15:08.064822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:15:08.064831 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:15:08.064917 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:15:08.064917 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:15:08.064935 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:15:08.065012 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:15:08.065194 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:08.065672 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:08.066501 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:08.066739 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:08.066802 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:08.067004 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:15:08.067110 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:15:08.067185 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 21:15:08.067263 (+ 78us) server_negotiation.cc:299] Negotiation successful
0924 21:15:08.067286 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 21:15:08.069355 22478 raft_consensus.cc:1261] T 1c3561677beb4ae09740e00943f7d295 P 8e270e70ec6a4095ae7b556b25bebbe4 [term 1 FOLLOWER]: Refusing update from remote peer 4100450f8fde416ba78f1bdb33a22f16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:15:08.069355 22614 raft_consensus.cc:1261] T 1c3561677beb4ae09740e00943f7d295 P 1eeca863d1c14311add655d8b95dce4b [term 1 FOLLOWER]: Refusing update from remote peer 4100450f8fde416ba78f1bdb33a22f16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:15:08.069775 22816 consensus_queue.cc:1035] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1eeca863d1c14311add655d8b95dce4b" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 42013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:15:08.070019 22821 consensus_queue.cc:1035] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:15:08.075124 22334 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47040:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:15:08.075213 22334 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47040:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:15:08.075977 22334 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b: Sending DeleteTablet for 3 replicas of tablet 1c3561677beb4ae09740e00943f7d295
I20260924 21:15:08.076253 22728 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c3561677beb4ae09740e00943f7d295 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:15:08 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:59452
I20260924 21:15:08.076295 22460 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c3561677beb4ae09740e00943f7d295 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:15:08 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:47838
I20260924 21:15:08.076313 22594 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c3561677beb4ae09740e00943f7d295 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:15:08 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:54080
I20260924 21:15:08.076512 22834 tablet_replica.cc:330] T 1c3561677beb4ae09740e00943f7d295 P 1eeca863d1c14311add655d8b95dce4b: stopping tablet replica
I20260924 21:15:08.076583 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 22397
I20260924 21:15:08.076608 22834 raft_consensus.cc:2229] T 1c3561677beb4ae09740e00943f7d295 P 1eeca863d1c14311add655d8b95dce4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:15:08.076709 22834 raft_consensus.cc:2258] T 1c3561677beb4ae09740e00943f7d295 P 1eeca863d1c14311add655d8b95dce4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:15:08.077155 22834 ts_tablet_manager.cc:1905] T 1c3561677beb4ae09740e00943f7d295 P 1eeca863d1c14311add655d8b95dce4b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:15:08.079173 22834 ts_tablet_manager.cc:1918] T 1c3561677beb4ae09740e00943f7d295 P 1eeca863d1c14311add655d8b95dce4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:15:08.079253 22834 log.cc:1196] T 1c3561677beb4ae09740e00943f7d295 P 1eeca863d1c14311add655d8b95dce4b: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/wal/wals/1c3561677beb4ae09740e00943f7d295
I20260924 21:15:08.079504 22834 ts_tablet_manager.cc:1939] T 1c3561677beb4ae09740e00943f7d295 P 1eeca863d1c14311add655d8b95dce4b: Deleting consensus metadata
I20260924 21:15:08.079916 22320 catalog_manager.cc:4841] TS 1eeca863d1c14311add655d8b95dce4b (127.6.210.130:42013): tablet 1c3561677beb4ae09740e00943f7d295 (table test-table [id=3b27794a09734938b654feb62b7e0238]) successfully deleted
I20260924 21:15:08.082407 22833 tablet_replica.cc:330] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16: stopping tablet replica
I20260924 21:15:08.082532 22833 raft_consensus.cc:2229] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16 [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:15:08.082670 22833 raft_consensus.cc:2258] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:15:08.083228 22833 ts_tablet_manager.cc:1905] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 21:15:08.083389 22321 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.210.129:32817 (error 108)
I20260924 21:15:08.083652 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 22531
I20260924 21:15:08.084200 22807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:08.083756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:32817 (local address 127.0.0.1:47850)
0924 21:15:08.084114 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:08.084162 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.129:32817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336}
W20260924 21:15:08.084738 22321 catalog_manager.cc:4584] TS 8e270e70ec6a4095ae7b556b25bebbe4 (127.6.210.129:32817): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1c3561677beb4ae09740e00943f7d295: Network error: Client connection negotiation failed: client connection to 127.6.210.129:32817: connect: Connection refused (error 111)
I20260924 21:15:08.086325 22833 ts_tablet_manager.cc:1918] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:15:08.086407 22833 log.cc:1196] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/wal/wals/1c3561677beb4ae09740e00943f7d295
I20260924 21:15:08.086696 22833 ts_tablet_manager.cc:1939] T 1c3561677beb4ae09740e00943f7d295 P 4100450f8fde416ba78f1bdb33a22f16: Deleting consensus metadata
I20260924 21:15:08.087414 22322 catalog_manager.cc:4841] TS 4100450f8fde416ba78f1bdb33a22f16 (127.6.210.131:38743): tablet 1c3561677beb4ae09740e00943f7d295 (table test-table [id=3b27794a09734938b654feb62b7e0238]) successfully deleted
I20260924 21:15:08.090950 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 22665
I20260924 21:15:08.097312 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 22303
I20260924 21:15:28.102979 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:36237
--webserver_interface=127.6.210.190
--webserver_port=34323
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:36793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:36237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:15:28.184559 22837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:15:28.184808 22837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:15:28.184844 22837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:15:28.185984 22837 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260924 21:15:28.186049 22837 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260924 21:15:28.186072 22837 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260924 21:15:28.186101 22837 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:15:28.186129 22837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:15:28.186157 22837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:15:28.186182 22837 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:15:28.186211 22837 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:15:28.187798 22837 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:36793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:36237
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:36237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=34323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:15:28.188068 22837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:15:28.188297 22837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:15:28.197461 22844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:15:28.197464 22846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:15:28.197547 22837 server_base.cc:1034] running on GCE node
W20260924 21:15:28.197464 22843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:15:28.197988 22837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:15:28.198441 22837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:15:28.199584 22837 hybrid_clock.cc:648] HybridClock initialized: now 1790284528199565 us; error 36 us; skew 500 ppm
Sep 24 21:15:28 dist-test-slave-q276 krb5kdc[22290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284528, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:15:28.202136 22837 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:15:28.202582 22837 webserver.cc:460] Webserver started at http://127.6.210.190:34323/ using document root <none> and password file <none>
I20260924 21:15:28.202800 22837 fs_manager.cc:359] Metadata directory not provided
I20260924 21:15:28.202852 22837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:15:28.204162 22837 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:15:28.204783 22853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:15:28.204962 22837 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:15:28.205044 22837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "cfb0af8fb6da4938802da89bde1d362b"
format_stamp: "Formatted at 2026-09-24 21:15:07 on dist-test-slave-q276"
server_key: "9a09c83fbe7f93b7f2ad83eb5b58731f"
server_key_iv: "84f522b76d313ced053fc82c1285b2cf"
server_key_version: "encryptionkey@0"
I20260924 21:15:28.205466 22837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:15:28.214358 22837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:15:28.216250 22837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:15:28.216389 22837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:15:28.220319 22837 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:36237
I20260924 21:15:28.220386 22905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:36237 every 8 connection(s)
I20260924 21:15:28.220762 22837 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:15:28.222251 22906 sys_catalog.cc:263] Verifying existing consensus state
I20260924 21:15:28.222950 22906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b: Bootstrap starting.
I20260924 21:15:28.227977 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 22837
I20260924 21:15:28.228286 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.129:32817
--local_ip_for_outbound_sockets=127.6.210.129
--tserver_master_addrs=127.6.210.190:36237
--webserver_port=37825
--webserver_interface=127.6.210.129
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:36793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:15:28.231726 22906 log.cc:824] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b: Log is configured to *not* fsync() on all Append() calls
I20260924 21:15:28.233609 22906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 21:15:28.233922 22906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b: Bootstrap complete.
I20260924 21:15:28.236212 22906 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb0af8fb6da4938802da89bde1d362b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36237 } }
I20260924 21:15:28.236459 22906 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfb0af8fb6da4938802da89bde1d362b, State: Initialized, Role: FOLLOWER
I20260924 21:15:28.236646 22906 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb0af8fb6da4938802da89bde1d362b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36237 } }
I20260924 21:15:28.236733 22906 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:15:28.236792 22906 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:15:28.236845 22906 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [term 1 FOLLOWER]: Advancing to term 2
I20260924 21:15:28.237916 22906 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb0af8fb6da4938802da89bde1d362b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36237 } }
I20260924 21:15:28.238039 22906 leader_election.cc:302] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfb0af8fb6da4938802da89bde1d362b; no voters:
I20260924 21:15:28.238240 22906 leader_election.cc:288] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 21:15:28.238299 22910 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [term 2 FOLLOWER]: Leader election won for term 2
I20260924 21:15:28.238420 22910 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [term 2 LEADER]: Becoming Leader. State: Replica: cfb0af8fb6da4938802da89bde1d362b, State: Running, Role: LEADER
I20260924 21:15:28.238529 22906 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:15:28.238569 22910 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb0af8fb6da4938802da89bde1d362b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36237 } }
I20260924 21:15:28.238926 22911 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "cfb0af8fb6da4938802da89bde1d362b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb0af8fb6da4938802da89bde1d362b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36237 } } }
I20260924 21:15:28.238924 22910 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfb0af8fb6da4938802da89bde1d362b. Latest consensus state: current_term: 2 leader_uuid: "cfb0af8fb6da4938802da89bde1d362b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb0af8fb6da4938802da89bde1d362b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36237 } } }
I20260924 21:15:28.239032 22911 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [sys.catalog]: This master's current role is: LEADER
I20260924 21:15:28.239055 22910 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [sys.catalog]: This master's current role is: LEADER
I20260924 21:15:28.239261 22918 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:15:28.240134 22918 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:15:28.240293 22918 catalog_manager.cc:1259] Loaded cluster ID: 690c7d21ead34b058754485a3d03cf3f
I20260924 21:15:28.240506 22918 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:15:28.241645 22918 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:15:28.241890 22918 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b: Loaded TSK: 0
I20260924 21:15:28.253425 22918 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b: Generated new TSK 1
I20260924 21:15:28.253540 22918 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 21:15:28.311964 22908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:15:28.312213 22908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:15:28.312255 22908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:15:28.313524 22908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:15:28.313591 22908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:15:28.313635 22908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.129
I20260924 21:15:28.315141 22908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:36793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.129:32817
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.210.129
--webserver_port=37825
--tserver_master_addrs=127.6.210.190:36237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.129
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:15:28.315399 22908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:15:28.315632 22908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:15:28.321488 22933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:15:28.321556 22936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:15:28.321501 22934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:15:28.321709 22908 server_base.cc:1034] running on GCE node
I20260924 21:15:28.322029 22908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:15:28.322340 22908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:15:28.323515 22908 hybrid_clock.cc:648] HybridClock initialized: now 1790284528323483 us; error 40 us; skew 500 ppm
Sep 24 21:15:28 dist-test-slave-q276 krb5kdc[22290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284528, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:15:28.331882 22908 init.cc:377] Logged in from keytab as kudu/127.6.210.129@KRBTEST.COM (short username kudu)
I20260924 21:15:28.332489 22908 webserver.cc:460] Webserver started at http://127.6.210.129:37825/ using document root <none> and password file <none>
I20260924 21:15:28.332759 22908 fs_manager.cc:359] Metadata directory not provided
I20260924 21:15:28.332850 22908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:15:28.335156 22908 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 21:15:28.336027 22943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:15:28.336256 22908 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:15:28.336364 22908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/wal
uuid: "8e270e70ec6a4095ae7b556b25bebbe4"
format_stamp: "Formatted at 2026-09-24 21:15:07 on dist-test-slave-q276"
server_key: "9b091536e0c8998f3636ccaca6cb0250"
server_key_iv: "2d1accd6050ce22132e01fe04e10dde7"
server_key_version: "encryptionkey@0"
I20260924 21:15:28.336814 22908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:15:28.349785 22908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:15:28.352180 22908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:15:28.352346 22908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:15:28.352605 22908 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:15:28.353075 22950 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 21:15:28.354776 22908 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 21:15:28.354838 22908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260924 21:15:28.354874 22908 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 21:15:28.355701 22908 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 21:15:28.355751 22908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260924 21:15:28.355839 22950 tablet_bootstrap.cc:492] T 1c3561677beb4ae09740e00943f7d295 P 8e270e70ec6a4095ae7b556b25bebbe4: Bootstrap starting.
I20260924 21:15:28.362782 22908 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.129:32817
I20260924 21:15:28.362910 23057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.129:32817 every 8 connection(s)
I20260924 21:15:28.363340 22908 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
I20260924 21:15:28.363653 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 22908
I20260924 21:15:28.363960 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.130:42013
--local_ip_for_outbound_sockets=127.6.210.130
--tserver_master_addrs=127.6.210.190:36237
--webserver_port=42323
--webserver_interface=127.6.210.130
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:36793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:15:28.366969 22950 log.cc:824] T 1c3561677beb4ae09740e00943f7d295 P 8e270e70ec6a4095ae7b556b25bebbe4: Log is configured to *not* fsync() on all Append() calls
Sep 24 21:15:28 dist-test-slave-q276 krb5kdc[22290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284528, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:15:28.372594 22950 tablet_bootstrap.cc:492] T 1c3561677beb4ae09740e00943f7d295 P 8e270e70ec6a4095ae7b556b25bebbe4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 21:15:28.373018 22950 tablet_bootstrap.cc:492] T 1c3561677beb4ae09740e00943f7d295 P 8e270e70ec6a4095ae7b556b25bebbe4: Bootstrap complete.
I20260924 21:15:28.374384 22950 ts_tablet_manager.cc:1397] T 1c3561677beb4ae09740e00943f7d295 P 8e270e70ec6a4095ae7b556b25bebbe4: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.010s
I20260924 21:15:28.375924 22950 raft_consensus.cc:348] T 1c3561677beb4ae09740e00943f7d295 P 8e270e70ec6a4095ae7b556b25bebbe4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4100450f8fde416ba78f1bdb33a22f16" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 38743 } } peers { permanent_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 32817 } } peers { permanent_uuid: "1eeca863d1c14311add655d8b95dce4b" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 42013 } }
I20260924 21:15:28.376382 22950 raft_consensus.cc:729] T 1c3561677beb4ae09740e00943f7d295 P 8e270e70ec6a4095ae7b556b25bebbe4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e270e70ec6a4095ae7b556b25bebbe4, State: Initialized, Role: FOLLOWER
I20260924 21:15:28.376602 22950 consensus_queue.cc:260] T 1c3561677beb4ae09740e00943f7d295 P 8e270e70ec6a4095ae7b556b25bebbe4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4100450f8fde416ba78f1bdb33a22f16" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 38743 } } peers { permanent_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 32817 } } peers { permanent_uuid: "1eeca863d1c14311add655d8b95dce4b" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 42013 } }
I20260924 21:15:28.376940 22950 ts_tablet_manager.cc:1428] T 1c3561677beb4ae09740e00943f7d295 P 8e270e70ec6a4095ae7b556b25bebbe4: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260924 21:15:28.377103 23060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:28.364251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.129:51353)
0924 21:15:28.364771 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:28.364815 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 21:15:28.365394 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:15:28.366062 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:15:28.366066 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:15:28.366453 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:15:28.366833 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:28.366839 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.368498 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.368500 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:28.369317 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:28.369319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.370332 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.370333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:28.370484 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:28.371092 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:15:28.371103 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:15:28.372821 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:15:28.375279 (+ 2458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:15:28.375283 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:15:28.375285 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:15:28.375562 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:15:28.375858 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:15:28.375859 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:15:28.375859 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:15:28.375980 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:15:28.376458 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:15:28.376463 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:15:28.376607 (+ 144us) client_negotiation.cc:769] Sending connection context
0924 21:15:28.376713 (+ 106us) client_negotiation.cc:240] Negotiation successful
0924 21:15:28.376903 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":86,"threads_started":1}
I20260924 21:15:28.377604 23061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:28.364583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:51353 (local address 127.6.210.190:36237)
0924 21:15:28.365589 (+ 1006us) server_negotiation.cc:206] Beginning negotiation
0924 21:15:28.365598 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:15:28.365633 (+ 35us) server_negotiation.cc:407] Connection header received
0924 21:15:28.365762 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:15:28.365768 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:15:28.365821 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:15:28.365938 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:15:28.366959 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.368379 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.369465 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.370086 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.370194 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:28.372960 (+ 2766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:15:28.372965 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:15:28.372965 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:15:28.372978 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:15:28.375183 (+ 2205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:15:28.375655 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:15:28.375658 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:15:28.375659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:15:28.375782 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:15:28.376055 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:15:28.376055 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:15:28.376056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:15:28.376349 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:15:28.376429 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0924 21:15:28.377368 (+ 939us) server_negotiation.cc:299] Negotiation successful
0924 21:15:28.377481 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":956,"thread_start_us":75,"threads_started":1}
I20260924 21:15:28.378250 23058 heartbeater.cc:347] Connected to a master server at 127.6.210.190:36237
I20260924 21:15:28.378366 23058 heartbeater.cc:464] Registering TS with master...
I20260924 21:15:28.378531 23058 heartbeater.cc:511] Master 127.6.210.190:36237 requested a full tablet report, sending...
I20260924 21:15:28.379235 22870 ts_manager.cc:194] Registered new tserver with Master: 8e270e70ec6a4095ae7b556b25bebbe4 (127.6.210.129:32817)
I20260924 21:15:28.380198 22870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.129@KRBTEST.COM'} at 127.6.210.129:51353
I20260924 21:15:28.381126 23058 heartbeater.cc:503] Master 127.6.210.190:36237 was elected leader, sending a full tablet report...
I20260924 21:15:28.389603 23066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:28.379865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39574 (local address 127.6.210.129:32817)
0924 21:15:28.382397 (+ 2532us) server_negotiation.cc:206] Beginning negotiation
0924 21:15:28.382401 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:15:28.386484 (+ 4083us) server_negotiation.cc:407] Connection header received
0924 21:15:28.386584 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:15:28.386587 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:15:28.386608 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:15:28.386691 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:15:28.386852 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.387501 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.388730 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.389192 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.389280 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:28.389338 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 21:15:28.389493 (+ 155us) server_negotiation.cc:299] Negotiation successful
0924 21:15:28.389532 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2502,"thread_start_us":73,"threads_started":1}
I20260924 21:15:28.389602 23065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:28.379805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:32817 (local address 127.0.0.1:39574)
0924 21:15:28.386375 (+ 6570us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:28.386409 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 21:15:28.386505 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:15:28.386694 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:15:28.386697 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:15:28.386699 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:15:28.386767 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:28.386769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.387587 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.387588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:28.388643 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:28.388645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.389283 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.389284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:28.389366 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:28.389378 (+ 12us) client_negotiation.cc:769] Sending connection context
0924 21:15:28.389466 (+ 88us) client_negotiation.cc:240] Negotiation successful
0924 21:15:28.389525 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6541,"thread_start_us":108,"threads_started":1}
I20260924 21:15:28.390223 22992 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c3561677beb4ae09740e00943f7d295 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:15:08 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:39574
I20260924 21:15:28.390769 23067 tablet_replica.cc:330] stopping tablet replica
I20260924 21:15:28.390897 23067 raft_consensus.cc:2229] T 1c3561677beb4ae09740e00943f7d295 P 8e270e70ec6a4095ae7b556b25bebbe4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:15:28.390969 23067 raft_consensus.cc:2258] T 1c3561677beb4ae09740e00943f7d295 P 8e270e70ec6a4095ae7b556b25bebbe4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:15:28.391279 23067 ts_tablet_manager.cc:1905] T 1c3561677beb4ae09740e00943f7d295 P 8e270e70ec6a4095ae7b556b25bebbe4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:15:28.393630 23067 ts_tablet_manager.cc:1918] T 1c3561677beb4ae09740e00943f7d295 P 8e270e70ec6a4095ae7b556b25bebbe4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:15:28.393743 23067 log.cc:1196] T 1c3561677beb4ae09740e00943f7d295 P 8e270e70ec6a4095ae7b556b25bebbe4: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/wal/wals/1c3561677beb4ae09740e00943f7d295
I20260924 21:15:28.393996 23067 ts_tablet_manager.cc:1939] T 1c3561677beb4ae09740e00943f7d295 P 8e270e70ec6a4095ae7b556b25bebbe4: Deleting consensus metadata
I20260924 21:15:28.394690 22856 catalog_manager.cc:4841] TS 8e270e70ec6a4095ae7b556b25bebbe4 (127.6.210.129:32817): tablet 1c3561677beb4ae09740e00943f7d295 (table test-table [id=3b27794a09734938b654feb62b7e0238]) successfully deleted
W20260924 21:15:28.456575 23062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:15:28.456794 23062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:15:28.456818 23062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:15:28.457927 23062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:15:28.457968 23062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:15:28.457995 23062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.130
I20260924 21:15:28.459415 23062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:36793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.130:42013
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.210.130
--webserver_port=42323
--tserver_master_addrs=127.6.210.190:36237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.130
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:15:28.459620 23062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:15:28.459801 23062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:15:28.465394 23072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:15:28.465385 23073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:15:28.465428 23062 server_base.cc:1034] running on GCE node
W20260924 21:15:28.465382 23075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:15:28.465999 23062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:15:28.466400 23062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:15:28.467568 23062 hybrid_clock.cc:648] HybridClock initialized: now 1790284528467557 us; error 30 us; skew 500 ppm
Sep 24 21:15:28 dist-test-slave-q276 krb5kdc[22290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284528, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:15:28.470189 23062 init.cc:377] Logged in from keytab as kudu/127.6.210.130@KRBTEST.COM (short username kudu)
I20260924 21:15:28.470731 23062 webserver.cc:460] Webserver started at http://127.6.210.130:42323/ using document root <none> and password file <none>
I20260924 21:15:28.470983 23062 fs_manager.cc:359] Metadata directory not provided
I20260924 21:15:28.471074 23062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:15:28.472506 23062 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:15:28.473249 23082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:15:28.473485 23062 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:15:28.473594 23062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/wal
uuid: "1eeca863d1c14311add655d8b95dce4b"
format_stamp: "Formatted at 2026-09-24 21:15:07 on dist-test-slave-q276"
server_key: "f2f4205afb85a38ff8b17056d9e1a2f7"
server_key_iv: "0bc83b69a687accadb72f95e15172476"
server_key_version: "encryptionkey@0"
I20260924 21:15:28.473960 23062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:15:28.485545 23062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:15:28.487612 23062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:15:28.487782 23062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:15:28.488025 23062 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:15:28.488386 23062 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:15:28.488433 23062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:15:28.488468 23062 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:15:28.488495 23062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:15:28.494632 23062 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.130:42013
I20260924 21:15:28.494674 23195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.130:42013 every 8 connection(s)
I20260924 21:15:28.495021 23062 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
Sep 24 21:15:28 dist-test-slave-q276 krb5kdc[22290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284528, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:15:28.501816 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 23062
I20260924 21:15:28.502101 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.131:38743
--local_ip_for_outbound_sockets=127.6.210.131
--tserver_master_addrs=127.6.210.190:36237
--webserver_port=42977
--webserver_interface=127.6.210.131
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:36793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:15:28.505219 23061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:28.495731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:38473 (local address 127.6.210.190:36237)
0924 21:15:28.495797 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 21:15:28.495800 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:15:28.496031 (+ 231us) server_negotiation.cc:407] Connection header received
0924 21:15:28.496668 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:15:28.496670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:15:28.496683 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:15:28.496730 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:15:28.497683 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.498186 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.499262 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.499454 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.499510 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:28.501959 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:15:28.501961 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:15:28.501961 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:15:28.501969 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:15:28.503756 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:15:28.504167 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:15:28.504169 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:15:28.504170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:15:28.504226 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:15:28.504524 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:15:28.504525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:15:28.504526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:15:28.504677 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:15:28.504740 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 21:15:28.505073 (+ 333us) server_negotiation.cc:299] Negotiation successful
0924 21:15:28.505153 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 21:15:28.505518 23198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:28.495745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.130:38473)
0924 21:15:28.495905 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:28.495944 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 21:15:28.496522 (+ 578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:15:28.496799 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:15:28.496802 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:15:28.497385 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:15:28.497581 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:28.497595 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.498334 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.498339 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:28.499159 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:28.499161 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.499552 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.499553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:28.499694 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:28.500380 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:15:28.500391 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:15:28.501830 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:15:28.503852 (+ 2022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:15:28.503859 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:15:28.503861 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:15:28.504088 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:15:28.504304 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:15:28.504305 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:15:28.504305 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:15:28.504438 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:15:28.504752 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:15:28.504759 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:15:28.504932 (+ 173us) client_negotiation.cc:769] Sending connection context
0924 21:15:28.505048 (+ 116us) client_negotiation.cc:240] Negotiation successful
0924 21:15:28.505236 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
I20260924 21:15:28.506059 23196 heartbeater.cc:347] Connected to a master server at 127.6.210.190:36237
I20260924 21:15:28.506156 23196 heartbeater.cc:464] Registering TS with master...
I20260924 21:15:28.506394 23196 heartbeater.cc:511] Master 127.6.210.190:36237 requested a full tablet report, sending...
I20260924 21:15:28.506794 22870 ts_manager.cc:194] Registered new tserver with Master: 1eeca863d1c14311add655d8b95dce4b (127.6.210.130:42013)
I20260924 21:15:28.507171 22870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.130@KRBTEST.COM'} at 127.6.210.130:38473
W20260924 21:15:28.585878 23199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:15:28.586169 23199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:15:28.586206 23199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:15:28.587745 23199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:15:28.587790 23199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:15:28.587817 23199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.131
I20260924 21:15:28.589322 23199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:36793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.131:38743
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.210.131
--webserver_port=42977
--tserver_master_addrs=127.6.210.190:36237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.131
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:15:28.589507 23199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:15:28.589676 23199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:15:28.594756 23204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:15:28.594887 23199 server_base.cc:1034] running on GCE node
W20260924 21:15:28.594910 23205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:15:28.594781 23207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:15:28.595160 23199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:15:28.595415 23199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:15:28.596580 23199 hybrid_clock.cc:648] HybridClock initialized: now 1790284528596567 us; error 29 us; skew 500 ppm
Sep 24 21:15:28 dist-test-slave-q276 krb5kdc[22290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284528, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:15:28.599293 23199 init.cc:377] Logged in from keytab as kudu/127.6.210.131@KRBTEST.COM (short username kudu)
I20260924 21:15:28.599714 23199 webserver.cc:460] Webserver started at http://127.6.210.131:42977/ using document root <none> and password file <none>
I20260924 21:15:28.599956 23199 fs_manager.cc:359] Metadata directory not provided
I20260924 21:15:28.600039 23199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:15:28.601387 23199 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:15:28.601974 23214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:15:28.602152 23199 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:15:28.602250 23199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/wal
uuid: "4100450f8fde416ba78f1bdb33a22f16"
format_stamp: "Formatted at 2026-09-24 21:15:07 on dist-test-slave-q276"
server_key: "90f1a61a32a75db61de042f759f25e54"
server_key_iv: "68a21c0e11bfe1df6aaf82846db47a26"
server_key_version: "encryptionkey@0"
I20260924 21:15:28.602694 23199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:15:28.641144 23199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:15:28.643502 23199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:15:28.643685 23199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:15:28.643962 23199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:15:28.644274 23199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:15:28.644333 23199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:15:28.644378 23199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:15:28.644402 23199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:15:28.650251 23199 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.131:38743
I20260924 21:15:28.650358 23327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.131:38743 every 8 connection(s)
I20260924 21:15:28.650734 23199 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
Sep 24 21:15:28 dist-test-slave-q276 krb5kdc[22290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284528, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:15:28.658165 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 23199
I20260924 21:15:28.661006 23061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:28.651560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:44733 (local address 127.6.210.190:36237)
0924 21:15:28.651621 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 21:15:28.651623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:15:28.651835 (+ 212us) server_negotiation.cc:407] Connection header received
0924 21:15:28.652301 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:15:28.652302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:15:28.652312 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:15:28.652375 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:15:28.653300 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.653803 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.654893 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.655079 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.655139 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:28.657279 (+ 2140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:15:28.657280 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:15:28.657280 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:15:28.657317 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:15:28.659501 (+ 2184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:15:28.660015 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:15:28.660016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:15:28.660016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:15:28.660073 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:15:28.660357 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:15:28.660358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:15:28.660358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:15:28.660460 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:15:28.660540 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0924 21:15:28.660822 (+ 282us) server_negotiation.cc:299] Negotiation successful
0924 21:15:28.660909 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 21:15:28.661201 23330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:28.651549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.131:44733)
0924 21:15:28.651694 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:28.651736 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 21:15:28.652212 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:15:28.652429 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:15:28.652433 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:15:28.653005 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:15:28.653199 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:28.653204 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.653897 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.653898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:28.654811 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:28.654813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.655157 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.655157 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:28.655255 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:28.655741 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:15:28.655750 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:15:28.657186 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:15:28.659589 (+ 2403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:15:28.659596 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:15:28.659598 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:15:28.659936 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:15:28.660140 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:15:28.660141 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:15:28.660141 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:15:28.660292 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:15:28.660532 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:15:28.660539 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:15:28.660691 (+ 152us) client_negotiation.cc:769] Sending connection context
0924 21:15:28.660783 (+ 92us) client_negotiation.cc:240] Negotiation successful
0924 21:15:28.660912 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":75,"threads_started":1}
I20260924 21:15:28.661721 23328 heartbeater.cc:347] Connected to a master server at 127.6.210.190:36237
I20260924 21:15:28.661844 23328 heartbeater.cc:464] Registering TS with master...
I20260924 21:15:28.662079 23328 heartbeater.cc:511] Master 127.6.210.190:36237 requested a full tablet report, sending...
I20260924 21:15:28.662593 22870 ts_manager.cc:194] Registered new tserver with Master: 4100450f8fde416ba78f1bdb33a22f16 (127.6.210.131:38743)
I20260924 21:15:28.663116 22870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.131@KRBTEST.COM'} at 127.6.210.131:44733
I20260924 21:15:28.666106 23332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:28.658638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50594 (local address 127.6.210.190:36237)
0924 21:15:28.658791 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0924 21:15:28.658793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:15:28.658806 (+ 13us) server_negotiation.cc:407] Connection header received
0924 21:15:28.658894 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:15:28.658895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:15:28.658906 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:15:28.658963 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:15:28.659729 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.660462 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.661507 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.661908 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.661993 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:28.663457 (+ 1464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:15:28.663458 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:15:28.663459 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:15:28.663467 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:15:28.664914 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:15:28.665245 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:15:28.665245 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:15:28.665246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:15:28.665316 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:15:28.665570 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:15:28.665572 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:15:28.665572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:15:28.665722 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:15:28.665868 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0924 21:15:28.665967 (+ 99us) server_negotiation.cc:299] Negotiation successful
0924 21:15:28.666039 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":99,"threads_started":1}
I20260924 21:15:28.666843 6986 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:15:28.669474 23332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:28.667184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50610 (local address 127.6.210.190:36237)
0924 21:15:28.667220 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 21:15:28.667222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:15:28.667401 (+ 179us) server_negotiation.cc:407] Connection header received
0924 21:15:28.667430 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:15:28.667431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:15:28.667459 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:15:28.667529 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:15:28.667647 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.668094 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.668847 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.669033 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.669087 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:28.669237 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:15:28.669278 (+ 41us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0924 21:15:28.669400 (+ 122us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50610: token expired
Metrics: {"server-negotiator.queue_time_us":19}
W20260924 21:15:28.669539 23332 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50610: token expired
W20260924 21:15:28.669628 22798 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.6.210.190:36237)
I20260924 21:15:28.675551 23332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:28.669858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50624 (local address 127.6.210.190:36237)
0924 21:15:28.669910 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 21:15:28.669912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:15:28.669941 (+ 29us) server_negotiation.cc:407] Connection header received
0924 21:15:28.669986 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:15:28.669987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:15:28.669994 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:15:28.670038 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:15:28.670564 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.670997 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.671666 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.671842 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.671899 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:28.673015 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:15:28.673016 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:15:28.673016 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:15:28.673022 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:15:28.674486 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:15:28.674865 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:15:28.674866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:15:28.674866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:15:28.674914 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:15:28.675064 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:15:28.675064 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:15:28.675064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:15:28.675194 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:15:28.675296 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0924 21:15:28.675426 (+ 130us) server_negotiation.cc:299] Negotiation successful
0924 21:15:28.675486 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 21:15:28.676903 22870 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50624:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:15:28.678689 22870 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:15:28.681123 22992 tablet_service.cc:1467] Processing CreateTablet for tablet 349a01fc3b7243019d0232d51af54366 (DEFAULT_TABLE table=test-table [id=a92bac71a55d49aab4024d53c5e9240a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:15:28.681329 22992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 349a01fc3b7243019d0232d51af54366. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:15:28.682860 22950 tablet_bootstrap.cc:492] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4: Bootstrap starting.
I20260924 21:15:28.683192 22950 tablet_bootstrap.cc:654] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4: Neither blocks nor log segments found. Creating new log.
I20260924 21:15:28.683862 22950 tablet_bootstrap.cc:492] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4: No bootstrap required, opened a new log
I20260924 21:15:28.683910 22950 ts_tablet_manager.cc:1397] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260924 21:15:28.684007 22950 raft_consensus.cc:348] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4100450f8fde416ba78f1bdb33a22f16" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 38743 } } peers { permanent_uuid: "1eeca863d1c14311add655d8b95dce4b" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 42013 } } peers { permanent_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 32817 } }
I20260924 21:15:28.684068 22950 raft_consensus.cc:374] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:15:28.684121 22950 raft_consensus.cc:729] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e270e70ec6a4095ae7b556b25bebbe4, State: Initialized, Role: FOLLOWER
I20260924 21:15:28.684183 22950 consensus_queue.cc:260] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4100450f8fde416ba78f1bdb33a22f16" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 38743 } } peers { permanent_uuid: "1eeca863d1c14311add655d8b95dce4b" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 42013 } } peers { permanent_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 32817 } }
I20260924 21:15:28.684283 22950 ts_tablet_manager.cc:1428] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:15:28.685302 23065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:28.680803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:38743 (local address 127.0.0.1:59778)
0924 21:15:28.680930 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:28.680942 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:15:28.680985 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:15:28.681476 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:15:28.681477 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:15:28.681479 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:15:28.681528 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:28.681530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.682945 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.682946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:28.684063 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:28.684066 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.685073 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.685074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:28.685135 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:28.685139 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:15:28.685184 (+ 45us) client_negotiation.cc:240] Negotiation successful
0924 21:15:28.685232 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106}
I20260924 21:15:28.685357 23336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:28.681088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59778 (local address 127.6.210.131:38743)
0924 21:15:28.681266 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0924 21:15:28.681272 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 21:15:28.681285 (+ 13us) server_negotiation.cc:407] Connection header received
0924 21:15:28.681329 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:15:28.681332 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:15:28.681349 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:15:28.681471 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:15:28.681599 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.682858 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.684165 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.684987 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.685075 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:28.685207 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0924 21:15:28.685236 (+ 29us) server_negotiation.cc:299] Negotiation successful
0924 21:15:28.685274 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":110,"threads_started":1}
I20260924 21:15:28.685958 23262 tablet_service.cc:1467] Processing CreateTablet for tablet 349a01fc3b7243019d0232d51af54366 (DEFAULT_TABLE table=test-table [id=a92bac71a55d49aab4024d53c5e9240a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:15:28.686306 23262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 349a01fc3b7243019d0232d51af54366. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:15:28.686456 23335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:28.681027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35338 (local address 127.6.210.130:42013)
0924 21:15:28.681322 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0924 21:15:28.681326 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:15:28.681335 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:15:28.681369 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:15:28.681371 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:15:28.681386 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:15:28.681611 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:15:28.681724 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.682857 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.685446 (+ 2589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.686051 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.686155 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:28.686213 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 21:15:28.686334 (+ 121us) server_negotiation.cc:299] Negotiation successful
0924 21:15:28.686372 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":84,"threads_started":1}
I20260924 21:15:28.686501 23334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:28.680839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:42013 (local address 127.0.0.1:35338)
0924 21:15:28.681015 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:28.681024 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:15:28.681064 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:15:28.681487 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:15:28.681487 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:15:28.681490 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:15:28.681537 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:28.681539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.684189 (+ 2650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.684190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:28.685329 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:28.685331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.686176 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.686177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:28.686242 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:28.686245 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:15:28.686319 (+ 74us) client_negotiation.cc:240] Negotiation successful
0924 21:15:28.686360 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":113,"threads_started":1}
I20260924 21:15:28.689005 23338 tablet_bootstrap.cc:492] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16: Bootstrap starting.
I20260924 21:15:28.689561 23130 tablet_service.cc:1467] Processing CreateTablet for tablet 349a01fc3b7243019d0232d51af54366 (DEFAULT_TABLE table=test-table [id=a92bac71a55d49aab4024d53c5e9240a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:15:28.689755 23338 tablet_bootstrap.cc:654] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16: Neither blocks nor log segments found. Creating new log.
I20260924 21:15:28.689862 23130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 349a01fc3b7243019d0232d51af54366. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:15:28.690378 23338 log.cc:824] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16: Log is configured to *not* fsync() on all Append() calls
I20260924 21:15:28.691139 23338 tablet_bootstrap.cc:492] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16: No bootstrap required, opened a new log
I20260924 21:15:28.691226 23338 ts_tablet_manager.cc:1397] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:15:28.692713 23340 tablet_bootstrap.cc:492] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b: Bootstrap starting.
I20260924 21:15:28.692945 23338 raft_consensus.cc:348] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4100450f8fde416ba78f1bdb33a22f16" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 38743 } } peers { permanent_uuid: "1eeca863d1c14311add655d8b95dce4b" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 42013 } } peers { permanent_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 32817 } }
I20260924 21:15:28.693146 23338 raft_consensus.cc:374] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:15:28.693202 23338 raft_consensus.cc:729] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4100450f8fde416ba78f1bdb33a22f16, State: Initialized, Role: FOLLOWER
I20260924 21:15:28.693249 23340 tablet_bootstrap.cc:654] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b: Neither blocks nor log segments found. Creating new log.
I20260924 21:15:28.693320 23338 consensus_queue.cc:260] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4100450f8fde416ba78f1bdb33a22f16" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 38743 } } peers { permanent_uuid: "1eeca863d1c14311add655d8b95dce4b" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 42013 } } peers { permanent_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 32817 } }
I20260924 21:15:28.693666 23328 heartbeater.cc:503] Master 127.6.210.190:36237 was elected leader, sending a full tablet report...
I20260924 21:15:28.693647 23338 ts_tablet_manager.cc:1428] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 21:15:28.693873 23340 log.cc:824] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b: Log is configured to *not* fsync() on all Append() calls
I20260924 21:15:28.694653 23340 tablet_bootstrap.cc:492] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b: No bootstrap required, opened a new log
I20260924 21:15:28.694738 23340 ts_tablet_manager.cc:1397] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:15:28.696293 23340 raft_consensus.cc:348] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4100450f8fde416ba78f1bdb33a22f16" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 38743 } } peers { permanent_uuid: "1eeca863d1c14311add655d8b95dce4b" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 42013 } } peers { permanent_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 32817 } }
I20260924 21:15:28.696457 23340 raft_consensus.cc:374] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:15:28.696519 23340 raft_consensus.cc:729] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1eeca863d1c14311add655d8b95dce4b, State: Initialized, Role: FOLLOWER
I20260924 21:15:28.696642 23340 consensus_queue.cc:260] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4100450f8fde416ba78f1bdb33a22f16" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 38743 } } peers { permanent_uuid: "1eeca863d1c14311add655d8b95dce4b" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 42013 } } peers { permanent_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 32817 } }
I20260924 21:15:28.696931 23340 ts_tablet_manager.cc:1428] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:15:28.697026 23196 heartbeater.cc:503] Master 127.6.210.190:36237 was elected leader, sending a full tablet report...
W20260924 21:15:28.745627 23197 tablet.cc:2369] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:15:28.803232 23343 raft_consensus.cc:482] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:15:28.803368 23343 raft_consensus.cc:504] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4100450f8fde416ba78f1bdb33a22f16" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 38743 } } peers { permanent_uuid: "1eeca863d1c14311add655d8b95dce4b" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 42013 } } peers { permanent_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 32817 } }
I20260924 21:15:28.803684 23343 leader_election.cc:288] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4100450f8fde416ba78f1bdb33a22f16 (127.6.210.131:38743), 8e270e70ec6a4095ae7b556b25bebbe4 (127.6.210.129:32817)
I20260924 21:15:28.806785 23066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:28.803984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:39265 (local address 127.6.210.129:32817)
0924 21:15:28.804031 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 21:15:28.804033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:15:28.804130 (+ 97us) server_negotiation.cc:407] Connection header received
0924 21:15:28.804202 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:15:28.804203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:15:28.804213 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:15:28.804272 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:15:28.804458 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.804980 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.805976 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.806425 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.806482 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:28.806510 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0924 21:15:28.806655 (+ 145us) server_negotiation.cc:299] Negotiation successful
0924 21:15:28.806703 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 21:15:28.806810 23336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:28.804036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:49621 (local address 127.6.210.131:38743)
0924 21:15:28.804083 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 21:15:28.804085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:15:28.804093 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:15:28.804123 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:15:28.804124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:15:28.804133 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:15:28.804190 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:15:28.804384 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.804888 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.805974 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.806436 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.806524 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:28.806533 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 21:15:28.806704 (+ 171us) server_negotiation.cc:299] Negotiation successful
0924 21:15:28.806742 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 21:15:28.806802 23344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:28.803874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:32817 (local address 127.6.210.130:39265)
0924 21:15:28.804045 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:28.804056 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:15:28.804123 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:15:28.804302 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:15:28.804302 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:15:28.804304 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:15:28.804387 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:28.804388 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.805068 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.805069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:28.805884 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:28.805887 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.806503 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.806505 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:28.806573 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:28.806577 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:15:28.806627 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 21:15:28.806658 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":15,"thread_start_us":90,"threads_started":1}
I20260924 21:15:28.806804 23198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:28.803866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:38743 (local address 127.6.210.130:49621)
0924 21:15:28.803933 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:28.803946 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:15:28.804007 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:15:28.804208 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:15:28.804209 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:15:28.804228 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:15:28.804306 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:28.804308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.804972 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.804973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:28.805882 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:28.805885 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.806538 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.806539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:28.806628 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:28.806631 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:15:28.806689 (+ 58us) client_negotiation.cc:240] Negotiation successful
0924 21:15:28.806723 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260924 21:15:28.807116 23012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "349a01fc3b7243019d0232d51af54366" candidate_uuid: "1eeca863d1c14311add655d8b95dce4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" is_pre_election: true
I20260924 21:15:28.807298 23012 raft_consensus.cc:2454] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1eeca863d1c14311add655d8b95dce4b in term 0.
I20260924 21:15:28.807474 23282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "349a01fc3b7243019d0232d51af54366" candidate_uuid: "1eeca863d1c14311add655d8b95dce4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4100450f8fde416ba78f1bdb33a22f16" is_pre_election: true
I20260924 21:15:28.807542 23085 leader_election.cc:302] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1eeca863d1c14311add655d8b95dce4b, 8e270e70ec6a4095ae7b556b25bebbe4; no voters:
I20260924 21:15:28.807627 23282 raft_consensus.cc:2454] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1eeca863d1c14311add655d8b95dce4b in term 0.
I20260924 21:15:28.807667 23343 raft_consensus.cc:2781] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:15:28.807791 23343 raft_consensus.cc:482] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:15:28.807826 23343 raft_consensus.cc:3037] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:15:28.808576 23343 raft_consensus.cc:504] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4100450f8fde416ba78f1bdb33a22f16" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 38743 } } peers { permanent_uuid: "1eeca863d1c14311add655d8b95dce4b" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 42013 } } peers { permanent_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 32817 } }
I20260924 21:15:28.808737 23343 leader_election.cc:288] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b [CANDIDATE]: Term 1 election: Requested vote from peers 4100450f8fde416ba78f1bdb33a22f16 (127.6.210.131:38743), 8e270e70ec6a4095ae7b556b25bebbe4 (127.6.210.129:32817)
I20260924 21:15:28.808871 23282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "349a01fc3b7243019d0232d51af54366" candidate_uuid: "1eeca863d1c14311add655d8b95dce4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4100450f8fde416ba78f1bdb33a22f16"
I20260924 21:15:28.808877 23012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "349a01fc3b7243019d0232d51af54366" candidate_uuid: "1eeca863d1c14311add655d8b95dce4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e270e70ec6a4095ae7b556b25bebbe4"
I20260924 21:15:28.808971 23282 raft_consensus.cc:3037] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:15:28.808974 23012 raft_consensus.cc:3037] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:15:28.809677 23282 raft_consensus.cc:2454] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1eeca863d1c14311add655d8b95dce4b in term 1.
I20260924 21:15:28.809854 23086 leader_election.cc:302] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1eeca863d1c14311add655d8b95dce4b, 4100450f8fde416ba78f1bdb33a22f16; no voters:
I20260924 21:15:28.810093 23012 raft_consensus.cc:2454] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1eeca863d1c14311add655d8b95dce4b in term 1.
I20260924 21:15:28.810201 23343 raft_consensus.cc:2781] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:15:28.810380 23343 raft_consensus.cc:686] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b [term 1 LEADER]: Becoming Leader. State: Replica: 1eeca863d1c14311add655d8b95dce4b, State: Running, Role: LEADER
I20260924 21:15:28.810513 23343 consensus_queue.cc:237] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4100450f8fde416ba78f1bdb33a22f16" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 38743 } } peers { permanent_uuid: "1eeca863d1c14311add655d8b95dce4b" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 42013 } } peers { permanent_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 32817 } }
I20260924 21:15:28.811352 22869 catalog_manager.cc:5495] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 1eeca863d1c14311add655d8b95dce4b (127.6.210.130). New cstate: current_term: 1 leader_uuid: "1eeca863d1c14311add655d8b95dce4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4100450f8fde416ba78f1bdb33a22f16" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 38743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1eeca863d1c14311add655d8b95dce4b" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 42013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 32817 } health_report { overall_health: UNKNOWN } } }
I20260924 21:15:28.834100 23335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:28.831856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35352 (local address 127.6.210.130:42013)
0924 21:15:28.831923 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 21:15:28.831925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:15:28.831936 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:15:28.831981 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:15:28.831982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:15:28.831999 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:15:28.832061 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:15:28.832219 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.832670 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.833377 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:28.833556 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:28.833605 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:28.833741 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:15:28.833846 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:15:28.833907 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 21:15:28.833942 (+ 35us) server_negotiation.cc:299] Negotiation successful
0924 21:15:28.833959 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 21:15:28.835988 23012 raft_consensus.cc:1261] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4 [term 1 FOLLOWER]: Refusing update from remote peer 1eeca863d1c14311add655d8b95dce4b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:15:28.836020 23282 raft_consensus.cc:1261] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16 [term 1 FOLLOWER]: Refusing update from remote peer 1eeca863d1c14311add655d8b95dce4b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:15:28.836390 23345 consensus_queue.cc:1035] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 32817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:15:28.836534 23343 consensus_queue.cc:1035] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4100450f8fde416ba78f1bdb33a22f16" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 38743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:15:28.838851 23348 mvcc.cc:205] Tried to move back new op lower bound from 7333005430109483008 to 7333005430008569856. Current Snapshot: MvccSnapshot[applied={T|T < 7333005430109483008}]
I20260924 21:15:28.839535 23350 mvcc.cc:205] Tried to move back new op lower bound from 7333005430109483008 to 7333005430008569856. Current Snapshot: MvccSnapshot[applied={T|T < 7333005430109483008}]
I20260924 21:15:28.841696 22870 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:15:28.841812 22870 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:15:28.842581 22870 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b: Sending DeleteTablet for 3 replicas of tablet 349a01fc3b7243019d0232d51af54366
I20260924 21:15:28.842909 23130 tablet_service.cc:1514] Processing DeleteTablet for tablet 349a01fc3b7243019d0232d51af54366 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:15:28 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:35338
I20260924 21:15:28.842967 23262 tablet_service.cc:1514] Processing DeleteTablet for tablet 349a01fc3b7243019d0232d51af54366 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:15:28 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:59778
I20260924 21:15:28.843152 23359 tablet_replica.cc:330] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16: stopping tablet replica
I20260924 21:15:28.843183 22992 tablet_service.cc:1514] Processing DeleteTablet for tablet 349a01fc3b7243019d0232d51af54366 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:15:28 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:39574
I20260924 21:15:28.843245 23359 raft_consensus.cc:2229] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:15:28.843276 23067 tablet_replica.cc:330] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4: stopping tablet replica
I20260924 21:15:28.843271 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 22908
I20260924 21:15:28.843317 23067 raft_consensus.cc:2229] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:15:28.843346 23359 raft_consensus.cc:2258] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:15:28.843362 23067 raft_consensus.cc:2258] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:15:28.843783 23359 ts_tablet_manager.cc:1905] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:15:28.845939 23359 ts_tablet_manager.cc:1918] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:15:28.846015 23359 log.cc:1196] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/wal/wals/349a01fc3b7243019d0232d51af54366
I20260924 21:15:28.846314 23359 ts_tablet_manager.cc:1939] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16: Deleting consensus metadata
I20260924 21:15:28.846736 22857 catalog_manager.cc:4841] TS 4100450f8fde416ba78f1bdb33a22f16 (127.6.210.131:38743): tablet 349a01fc3b7243019d0232d51af54366 (table test-table [id=a92bac71a55d49aab4024d53c5e9240a]) successfully deleted
I20260924 21:15:28.850418 23358 tablet_replica.cc:330] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b: stopping tablet replica
W20260924 21:15:28.850533 22856 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.210.129:32817 (error 108)
I20260924 21:15:28.850644 23358 raft_consensus.cc:2229] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:15:28.850798 23358 raft_consensus.cc:2258] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:15:28.850934 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 23062
I20260924 21:15:28.852371 23334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:28.851427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:32817 (local address 127.0.0.1:39580)
0924 21:15:28.852291 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:28.852337 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.129:32817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":819}
W20260924 21:15:28.852617 22856 catalog_manager.cc:4584] TS 8e270e70ec6a4095ae7b556b25bebbe4 (127.6.210.129:32817): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 349a01fc3b7243019d0232d51af54366: Network error: Client connection negotiation failed: client connection to 127.6.210.129:32817: connect: Connection refused (error 111)
I20260924 21:15:28.856900 23334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:28.856748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:42013 (local address 127.0.0.1:35356)
0924 21:15:28.856823 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:28.856857 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.130:42013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260924 21:15:28.856928 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 23199
I20260924 21:15:28.861722 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 22837
I20260924 21:15:48.867255 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:36237
--webserver_interface=127.6.210.190
--webserver_port=34323
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:36793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:36237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:15:48.952944 23361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:15:48.953161 23361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:15:48.953184 23361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:15:48.954319 23361 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260924 21:15:48.954357 23361 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260924 21:15:48.954372 23361 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260924 21:15:48.954385 23361 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:15:48.954401 23361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:15:48.954416 23361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:15:48.954429 23361 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:15:48.954443 23361 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:15:48.955858 23361 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:36793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:36237
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:36237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=34323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:15:48.956053 23361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:15:48.956235 23361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:15:48.961404 23366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:15:48.961406 23367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:15:48.961583 23361 server_base.cc:1034] running on GCE node
W20260924 21:15:48.961503 23369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:15:48.961978 23361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:15:48.962347 23361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:15:48.963552 23361 hybrid_clock.cc:648] HybridClock initialized: now 1790284548963543 us; error 44 us; skew 500 ppm
Sep 24 21:15:48 dist-test-slave-q276 krb5kdc[22290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284548, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:15:48.970098 23361 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:15:48.970656 23361 webserver.cc:460] Webserver started at http://127.6.210.190:34323/ using document root <none> and password file <none>
I20260924 21:15:48.970937 23361 fs_manager.cc:359] Metadata directory not provided
I20260924 21:15:48.971012 23361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:15:48.972415 23361 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 21:15:48.973147 23376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:15:48.973443 23361 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:15:48.973574 23361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "cfb0af8fb6da4938802da89bde1d362b"
format_stamp: "Formatted at 2026-09-24 21:15:07 on dist-test-slave-q276"
server_key: "9a09c83fbe7f93b7f2ad83eb5b58731f"
server_key_iv: "84f522b76d313ced053fc82c1285b2cf"
server_key_version: "encryptionkey@0"
I20260924 21:15:48.973956 23361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:15:48.991925 23361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:15:48.994454 23361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:15:48.994627 23361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:15:48.999050 23361 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:36237
I20260924 21:15:48.999091 23428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:36237 every 8 connection(s)
I20260924 21:15:48.999642 23361 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:15:49.000962 23429 sys_catalog.cc:263] Verifying existing consensus state
I20260924 21:15:49.001677 23429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b: Bootstrap starting.
I20260924 21:15:49.003109 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 23361
I20260924 21:15:49.003378 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.129:32817
--local_ip_for_outbound_sockets=127.6.210.129
--tserver_master_addrs=127.6.210.190:36237
--webserver_port=37825
--webserver_interface=127.6.210.129
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:36793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:15:49.010342 23429 log.cc:824] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b: Log is configured to *not* fsync() on all Append() calls
I20260924 21:15:49.012552 23429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 21:15:49.012898 23429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b: Bootstrap complete.
I20260924 21:15:49.015215 23429 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb0af8fb6da4938802da89bde1d362b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36237 } }
I20260924 21:15:49.015461 23429 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfb0af8fb6da4938802da89bde1d362b, State: Initialized, Role: FOLLOWER
I20260924 21:15:49.015642 23429 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb0af8fb6da4938802da89bde1d362b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36237 } }
I20260924 21:15:49.015724 23429 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:15:49.015782 23429 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:15:49.015830 23429 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [term 2 FOLLOWER]: Advancing to term 3
I20260924 21:15:49.016791 23429 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb0af8fb6da4938802da89bde1d362b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36237 } }
I20260924 21:15:49.016913 23429 leader_election.cc:302] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfb0af8fb6da4938802da89bde1d362b; no voters:
I20260924 21:15:49.017114 23429 leader_election.cc:288] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [CANDIDATE]: Term 3 election: Requested vote from peers
I20260924 21:15:49.017149 23433 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [term 3 FOLLOWER]: Leader election won for term 3
I20260924 21:15:49.017275 23433 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [term 3 LEADER]: Becoming Leader. State: Replica: cfb0af8fb6da4938802da89bde1d362b, State: Running, Role: LEADER
I20260924 21:15:49.017380 23429 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:15:49.017359 23433 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb0af8fb6da4938802da89bde1d362b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36237 } }
I20260924 21:15:49.017630 23433 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfb0af8fb6da4938802da89bde1d362b. Latest consensus state: current_term: 3 leader_uuid: "cfb0af8fb6da4938802da89bde1d362b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb0af8fb6da4938802da89bde1d362b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36237 } } }
I20260924 21:15:49.017695 23433 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [sys.catalog]: This master's current role is: LEADER
I20260924 21:15:49.017863 23434 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "cfb0af8fb6da4938802da89bde1d362b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb0af8fb6da4938802da89bde1d362b" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36237 } } }
I20260924 21:15:49.017937 23434 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b [sys.catalog]: This master's current role is: LEADER
I20260924 21:15:49.018416 23437 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:15:49.019192 23437 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:15:49.019311 23437 catalog_manager.cc:1259] Loaded cluster ID: 690c7d21ead34b058754485a3d03cf3f
I20260924 21:15:49.019348 23437 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:15:49.020460 23437 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:15:49.020641 23437 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b: Loaded TSK: 0
I20260924 21:15:49.020690 23437 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cfb0af8fb6da4938802da89bde1d362b: Loaded TSK: 1
F20260924 21:15:49.020918 23437 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f893572f9ad google::LogMessage::Fail() at ??:0
@ 0x7f893573288a google::LogMessage::SendToLog() at ??:0
@ 0x7f893572f4de google::LogMessage::Flush() at ??:0
@ 0x7f8935730a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f8937e9c104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f8937e99d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f893d68d1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f893d68d48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f893d68d594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f893d653c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f893d65dad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f89375fd0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f89375f2954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f89384566db start_thread at ??:0
@ 0x7f893422d61f clone at ??:0
W20260924 21:15:49.103752 23431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:15:49.103950 23431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:15:49.103973 23431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:15:49.105149 23431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:15:49.105191 23431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:15:49.105217 23431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.129
I20260924 21:15:49.106709 23431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:36793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.129:32817
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.210.129
--webserver_port=37825
--tserver_master_addrs=127.6.210.190:36237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.129
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:15:49.106940 23431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:15:49.107179 23431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:15:49.113819 23471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:15:49.113786 23473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:15:49.113786 23470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:15:49.114038 23431 server_base.cc:1034] running on GCE node
I20260924 21:15:49.114373 23431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:15:49.114627 23431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:15:49.115797 23431 hybrid_clock.cc:648] HybridClock initialized: now 1790284549115789 us; error 38 us; skew 500 ppm
Sep 24 21:15:49 dist-test-slave-q276 krb5kdc[22290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284549, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:15:49.118549 23431 init.cc:377] Logged in from keytab as kudu/127.6.210.129@KRBTEST.COM (short username kudu)
I20260924 21:15:49.119064 23431 webserver.cc:460] Webserver started at http://127.6.210.129:37825/ using document root <none> and password file <none>
I20260924 21:15:49.119297 23431 fs_manager.cc:359] Metadata directory not provided
I20260924 21:15:49.119342 23431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:15:49.120702 23431 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:15:49.121402 23480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:15:49.121577 23431 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:15:49.121658 23431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/wal
uuid: "8e270e70ec6a4095ae7b556b25bebbe4"
format_stamp: "Formatted at 2026-09-24 21:15:07 on dist-test-slave-q276"
server_key: "9b091536e0c8998f3636ccaca6cb0250"
server_key_iv: "2d1accd6050ce22132e01fe04e10dde7"
server_key_version: "encryptionkey@0"
I20260924 21:15:49.122017 23431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:15:49.137233 23431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:15:49.139695 23431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:15:49.139840 23431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:15:49.140112 23431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:15:49.140684 23487 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 21:15:49.142925 23431 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 21:15:49.142988 23431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260924 21:15:49.143030 23431 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 21:15:49.143759 23431 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 21:15:49.143823 23431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260924 21:15:49.143903 23487 tablet_bootstrap.cc:492] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4: Bootstrap starting.
I20260924 21:15:49.150906 23431 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.129:32817
I20260924 21:15:49.151304 23431 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
I20260924 21:15:49.151877 23594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.129:32817 every 8 connection(s)
I20260924 21:15:49.154924 23487 log.cc:824] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4: Log is configured to *not* fsync() on all Append() calls
I20260924 21:15:49.157745 23487 tablet_bootstrap.cc:492] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 21:15:49.158154 23487 tablet_bootstrap.cc:492] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4: Bootstrap complete.
I20260924 21:15:49.158736 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 23431
I20260924 21:15:49.159029 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.130:42013
--local_ip_for_outbound_sockets=127.6.210.130
--tserver_master_addrs=127.6.210.190:36237
--webserver_port=42323
--webserver_interface=127.6.210.130
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:36793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:15:49.159427 23487 ts_tablet_manager.cc:1397] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.007s
I20260924 21:15:49.160979 23487 raft_consensus.cc:348] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4100450f8fde416ba78f1bdb33a22f16" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 38743 } } peers { permanent_uuid: "1eeca863d1c14311add655d8b95dce4b" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 42013 } } peers { permanent_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 32817 } }
I20260924 21:15:49.161351 23487 raft_consensus.cc:729] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e270e70ec6a4095ae7b556b25bebbe4, State: Initialized, Role: FOLLOWER
I20260924 21:15:49.161522 23487 consensus_queue.cc:260] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4100450f8fde416ba78f1bdb33a22f16" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 38743 } } peers { permanent_uuid: "1eeca863d1c14311add655d8b95dce4b" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 42013 } } peers { permanent_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 32817 } }
I20260924 21:15:49.161868 23487 ts_tablet_manager.cc:1428] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
W20260924 21:15:49.225174 23597 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:15:49.152061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.129:44315)
0924 21:15:49.152229 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:49.152271 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 21:15:49.152737 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:15:49.224955 (+ 72218us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":103,"threads_started":1}
W20260924 21:15:49.225503 23595 heartbeater.cc:644] Failed to heartbeat to 127.6.210.190:36237 (0 consecutive failures): Network error: Failed to ping master at 127.6.210.190:36237: Client connection negotiation failed: client connection to 127.6.210.190:36237: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 21:15:49.225917 23597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:49.225785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.129:46171)
0924 21:15:49.225842 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:49.225891 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260924 21:15:49.226301 23597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:49.226178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.129:42433)
0924 21:15:49.226236 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:49.226280 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260924 21:15:49.226428 23595 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 21:15:49.250113 23599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:15:49.250406 23599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:15:49.250466 23599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:15:49.251718 23599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:15:49.251801 23599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:15:49.251847 23599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.130
I20260924 21:15:49.253319 23599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:36793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.130:42013
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.210.130
--webserver_port=42323
--tserver_master_addrs=127.6.210.190:36237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.130
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:15:49.253587 23599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:15:49.253829 23599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 21:15:49.259001 23599 server_base.cc:1034] running on GCE node
W20260924 21:15:49.258968 23605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:15:49.258944 23608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:15:49.258944 23606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:15:49.259421 23599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:15:49.259632 23599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:15:49.260794 23599 hybrid_clock.cc:648] HybridClock initialized: now 1790284549260785 us; error 30 us; skew 500 ppm
Sep 24 21:15:49 dist-test-slave-q276 krb5kdc[22290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284549, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:15:49.263566 23599 init.cc:377] Logged in from keytab as kudu/127.6.210.130@KRBTEST.COM (short username kudu)
I20260924 21:15:49.263954 23599 webserver.cc:460] Webserver started at http://127.6.210.130:42323/ using document root <none> and password file <none>
I20260924 21:15:49.264178 23599 fs_manager.cc:359] Metadata directory not provided
I20260924 21:15:49.264254 23599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:15:49.265545 23599 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:15:49.266165 23615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:15:49.266390 23599 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:15:49.266485 23599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/wal
uuid: "1eeca863d1c14311add655d8b95dce4b"
format_stamp: "Formatted at 2026-09-24 21:15:07 on dist-test-slave-q276"
server_key: "f2f4205afb85a38ff8b17056d9e1a2f7"
server_key_iv: "0bc83b69a687accadb72f95e15172476"
server_key_version: "encryptionkey@0"
I20260924 21:15:49.266856 23599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:15:49.276283 23599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:15:49.278761 23599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:15:49.278932 23599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:15:49.279165 23599 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:15:49.279601 23622 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 21:15:49.281085 23599 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 21:15:49.281178 23599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260924 21:15:49.281311 23599 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 21:15:49.282014 23599 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 21:15:49.282084 23599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260924 21:15:49.282130 23622 tablet_bootstrap.cc:492] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b: Bootstrap starting.
I20260924 21:15:49.289513 23599 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.130:42013
I20260924 21:15:49.289621 23729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.130:42013 every 8 connection(s)
I20260924 21:15:49.289965 23599 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
I20260924 21:15:49.291148 23732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:49.290716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.130:36511)
0924 21:15:49.290945 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:49.291022 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":115,"threads_started":1}
W20260924 21:15:49.291473 23730 heartbeater.cc:644] Failed to heartbeat to 127.6.210.190:36237 (0 consecutive failures): Network error: Failed to ping master at 127.6.210.190:36237: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
I20260924 21:15:49.291816 23732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:49.291690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.130:47307)
0924 21:15:49.291761 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:49.291788 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260924 21:15:49.292145 23732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:49.292048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.130:37549)
0924 21:15:49.292099 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:49.292124 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260924 21:15:49.292272 23730 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 21:15:49.292959 23622 log.cc:824] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b: Log is configured to *not* fsync() on all Append() calls
I20260924 21:15:49.294709 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 23599
I20260924 21:15:49.294978 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.131:38743
--local_ip_for_outbound_sockets=127.6.210.131
--tserver_master_addrs=127.6.210.190:36237
--webserver_port=42977
--webserver_interface=127.6.210.131
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:36793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:15:49.295756 23622 tablet_bootstrap.cc:492] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 21:15:49.296123 23622 tablet_bootstrap.cc:492] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b: Bootstrap complete.
I20260924 21:15:49.297549 23622 ts_tablet_manager.cc:1397] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.008s
I20260924 21:15:49.300118 23622 raft_consensus.cc:348] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4100450f8fde416ba78f1bdb33a22f16" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 38743 } } peers { permanent_uuid: "1eeca863d1c14311add655d8b95dce4b" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 42013 } } peers { permanent_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 32817 } }
I20260924 21:15:49.300586 23622 raft_consensus.cc:729] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1eeca863d1c14311add655d8b95dce4b, State: Initialized, Role: FOLLOWER
I20260924 21:15:49.300786 23622 consensus_queue.cc:260] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4100450f8fde416ba78f1bdb33a22f16" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 38743 } } peers { permanent_uuid: "1eeca863d1c14311add655d8b95dce4b" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 42013 } } peers { permanent_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 32817 } }
I20260924 21:15:49.301136 23622 ts_tablet_manager.cc:1428] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b: Time spent starting tablet: real 0.004s user 0.007s sys 0.000s
I20260924 21:15:49.301606 23732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:49.301455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.130:56675)
0924 21:15:49.301524 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:49.301566 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260924 21:15:49.302006 23732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:49.301881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.130:49487)
0924 21:15:49.301940 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:49.301974 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260924 21:15:49.386085 23734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:15:49.386339 23734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:15:49.386368 23734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:15:49.388263 23734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:15:49.388317 23734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:15:49.388356 23734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.131
I20260924 21:15:49.389973 23734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:36793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.131:38743
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.210.131
--webserver_port=42977
--tserver_master_addrs=127.6.210.190:36237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.131
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:15:49.390249 23734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:15:49.390508 23734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:15:49.396102 23741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:15:49.396107 23743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:15:49.396129 23740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:15:49.396646 23734 server_base.cc:1034] running on GCE node
I20260924 21:15:49.396857 23734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:15:49.397118 23734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:15:49.398350 23734 hybrid_clock.cc:648] HybridClock initialized: now 1790284549398329 us; error 30 us; skew 500 ppm
Sep 24 21:15:49 dist-test-slave-q276 krb5kdc[22290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284549, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:15:49.401036 23734 init.cc:377] Logged in from keytab as kudu/127.6.210.131@KRBTEST.COM (short username kudu)
I20260924 21:15:49.401657 23734 webserver.cc:460] Webserver started at http://127.6.210.131:42977/ using document root <none> and password file <none>
I20260924 21:15:49.401965 23734 fs_manager.cc:359] Metadata directory not provided
I20260924 21:15:49.402055 23734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:15:49.403600 23734 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:15:49.404362 23750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:15:49.404587 23734 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:15:49.404695 23734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/wal
uuid: "4100450f8fde416ba78f1bdb33a22f16"
format_stamp: "Formatted at 2026-09-24 21:15:07 on dist-test-slave-q276"
server_key: "90f1a61a32a75db61de042f759f25e54"
server_key_iv: "68a21c0e11bfe1df6aaf82846db47a26"
server_key_version: "encryptionkey@0"
I20260924 21:15:49.405068 23734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:15:49.417651 23734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:15:49.419816 23734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:15:49.419993 23734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:15:49.420224 23734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:15:49.420574 23734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:15:49.420621 23734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:15:49.420655 23734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:15:49.420682 23734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:15:49.427098 23734 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.131:38743
I20260924 21:15:49.427230 23863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.131:38743 every 8 connection(s)
I20260924 21:15:49.427554 23734 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
I20260924 21:15:49.429113 23866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:49.428500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.131:35637)
0924 21:15:49.428712 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:49.428854 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":93,"threads_started":1}
W20260924 21:15:49.429899 23864 heartbeater.cc:644] Failed to heartbeat to 127.6.210.190:36237 (0 consecutive failures): Network error: Failed to ping master at 127.6.210.190:36237: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
I20260924 21:15:49.430253 23866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:49.430141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.131:40667)
0924 21:15:49.430202 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:49.430232 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260924 21:15:49.430648 23866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:49.430542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.131:57833)
0924 21:15:49.430600 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:49.430629 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260924 21:15:49.430750 23864 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 21:15:49.430796 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 23734
I20260924 21:15:50.227056 23868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:50.226793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.129:58367)
0924 21:15:50.226951 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:50.226995 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":74,"threads_started":1}
I20260924 21:15:50.302675 23869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:50.302467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.130:47091)
0924 21:15:50.302586 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:50.302620 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260924 21:15:50.431284 23870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:50.431062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.131:60487)
0924 21:15:50.431184 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:50.431222 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":54,"threads_started":1}
I20260924 21:15:50.517304 23871 raft_consensus.cc:482] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:15:50.517436 23871 raft_consensus.cc:504] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4100450f8fde416ba78f1bdb33a22f16" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 38743 } } peers { permanent_uuid: "1eeca863d1c14311add655d8b95dce4b" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 42013 } } peers { permanent_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 32817 } }
I20260924 21:15:50.517874 23871 leader_election.cc:288] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4100450f8fde416ba78f1bdb33a22f16 (127.6.210.131:38743), 1eeca863d1c14311add655d8b95dce4b (127.6.210.130:42013)
Sep 24 21:15:50 dist-test-slave-q276 krb5kdc[22290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284549, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for kudu/127.6.210.131@KRBTEST.COM
Sep 24 21:15:50 dist-test-slave-q276 krb5kdc[22290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284549, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for kudu/127.6.210.130@KRBTEST.COM
I20260924 21:15:50.528406 23874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:50.518233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:37099 (local address 127.6.210.131:38743)
0924 21:15:50.518391 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0924 21:15:50.518405 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0924 21:15:50.518434 (+ 29us) server_negotiation.cc:407] Connection header received
0924 21:15:50.518844 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:15:50.518846 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:15:50.518878 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:15:50.519031 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:15:50.519379 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:50.520223 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:50.521366 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:50.521754 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:50.521887 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:50.524107 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:15:50.524112 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:15:50.524112 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:15:50.524124 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:15:50.526527 (+ 2403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:15:50.526997 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:15:50.526999 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:15:50.526999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:15:50.527079 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:15:50.527347 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:15:50.527348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:15:50.527349 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:15:50.527677 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:15:50.527759 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0924 21:15:50.528162 (+ 403us) server_negotiation.cc:299] Negotiation successful
0924 21:15:50.528295 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":77,"threads_started":1}
I20260924 21:15:50.528441 23868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:50.518065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:42013 (local address 127.6.210.129:34187)
0924 21:15:50.518114 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:50.518127 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:15:50.518192 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:15:50.518753 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:15:50.518756 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:15:50.519075 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:15:50.519257 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:50.519263 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:50.520353 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:50.520354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:50.521291 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:50.521294 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:50.521927 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:50.521928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:50.521988 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:50.522748 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:15:50.522754 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:15:50.524597 (+ 1843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:15:50.526927 (+ 2330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:15:50.526928 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:15:50.526928 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:15:50.527099 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:15:50.527389 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:15:50.527390 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:15:50.527390 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:15:50.527438 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:15:50.527848 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:15:50.527848 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:15:50.527947 (+ 99us) client_negotiation.cc:769] Sending connection context
0924 21:15:50.528082 (+ 135us) client_negotiation.cc:240] Negotiation successful
0924 21:15:50.528303 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":95}
I20260924 21:15:50.528584 23872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:50.518089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:38743 (local address 127.6.210.129:37099)
0924 21:15:50.518234 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:50.518245 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:15:50.518320 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:15:50.519010 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:15:50.519011 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:15:50.519201 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:15:50.519259 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:50.519266 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:50.520332 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:50.520334 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:50.521277 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:50.521280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:50.521860 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:50.521861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:50.521967 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:50.522549 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:15:50.522558 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:15:50.523995 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:15:50.526622 (+ 2627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:15:50.526628 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:15:50.526630 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:15:50.526902 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:15:50.527149 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:15:50.527150 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:15:50.527151 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:15:50.527278 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:15:50.527771 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:15:50.527776 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:15:50.527947 (+ 171us) client_negotiation.cc:769] Sending connection context
0924 21:15:50.528081 (+ 134us) client_negotiation.cc:240] Negotiation successful
0924 21:15:50.528392 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":153,"thread_start_us":78,"threads_started":1}
I20260924 21:15:50.528398 23873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:50.518182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:34187 (local address 127.6.210.130:42013)
0924 21:15:50.518355 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0924 21:15:50.518367 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 21:15:50.518391 (+ 24us) server_negotiation.cc:407] Connection header received
0924 21:15:50.518521 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:15:50.518523 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:15:50.518562 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:15:50.518701 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:15:50.519375 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:50.520241 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:50.521400 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:50.521830 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:50.521972 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:50.524717 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:15:50.524721 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:15:50.524726 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:15:50.524736 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:15:50.526838 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:15:50.527200 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:15:50.527202 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:15:50.527203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:15:50.527294 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:15:50.527504 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:15:50.527505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:15:50.527505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:15:50.527774 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:15:50.527843 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 21:15:50.528162 (+ 319us) server_negotiation.cc:299] Negotiation successful
0924 21:15:50.528295 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":56,"threads_started":1}
W20260924 21:15:50.529372 23484 leader_election.cc:341] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4100450f8fde416ba78f1bdb33a22f16 (127.6.210.131:38743): Not found: Tablet not found: 349a01fc3b7243019d0232d51af54366
I20260924 21:15:50.529350 23684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "349a01fc3b7243019d0232d51af54366" candidate_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1eeca863d1c14311add655d8b95dce4b" is_pre_election: true
I20260924 21:15:50.529522 23684 raft_consensus.cc:2454] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e270e70ec6a4095ae7b556b25bebbe4 in term 1.
I20260924 21:15:50.529753 23482 leader_election.cc:302] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1eeca863d1c14311add655d8b95dce4b, 8e270e70ec6a4095ae7b556b25bebbe4; no voters: 4100450f8fde416ba78f1bdb33a22f16
I20260924 21:15:50.529902 23871 raft_consensus.cc:2781] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 21:15:50.529981 23871 raft_consensus.cc:482] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:15:50.530018 23871 raft_consensus.cc:3037] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4 [term 1 FOLLOWER]: Advancing to term 2
I20260924 21:15:50.532949 23871 raft_consensus.cc:504] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4100450f8fde416ba78f1bdb33a22f16" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 38743 } } peers { permanent_uuid: "1eeca863d1c14311add655d8b95dce4b" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 42013 } } peers { permanent_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 32817 } }
I20260924 21:15:50.533114 23871 leader_election.cc:288] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4 [CANDIDATE]: Term 2 election: Requested vote from peers 4100450f8fde416ba78f1bdb33a22f16 (127.6.210.131:38743), 1eeca863d1c14311add655d8b95dce4b (127.6.210.130:42013)
I20260924 21:15:50.533367 23684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "349a01fc3b7243019d0232d51af54366" candidate_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1eeca863d1c14311add655d8b95dce4b"
W20260924 21:15:50.533416 23484 leader_election.cc:341] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 4100450f8fde416ba78f1bdb33a22f16 (127.6.210.131:38743): Not found: Tablet not found: 349a01fc3b7243019d0232d51af54366
I20260924 21:15:50.533457 23684 raft_consensus.cc:3037] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b [term 1 FOLLOWER]: Advancing to term 2
I20260924 21:15:50.534600 23684 raft_consensus.cc:2454] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e270e70ec6a4095ae7b556b25bebbe4 in term 2.
I20260924 21:15:50.534802 23482 leader_election.cc:302] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1eeca863d1c14311add655d8b95dce4b, 8e270e70ec6a4095ae7b556b25bebbe4; no voters: 4100450f8fde416ba78f1bdb33a22f16
I20260924 21:15:50.534914 23871 raft_consensus.cc:2781] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 21:15:50.535022 23871 raft_consensus.cc:686] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4 [term 2 LEADER]: Becoming Leader. State: Replica: 8e270e70ec6a4095ae7b556b25bebbe4, State: Running, Role: LEADER
I20260924 21:15:50.535132 23871 consensus_queue.cc:237] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4100450f8fde416ba78f1bdb33a22f16" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 38743 } } peers { permanent_uuid: "1eeca863d1c14311add655d8b95dce4b" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 42013 } } peers { permanent_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 32817 } }
I20260924 21:15:50.535414 23872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:50.535225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.129:40835)
0924 21:15:50.535348 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:50.535380 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
W20260924 21:15:50.944794 23484 consensus_peers.cc:487] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4 -> Peer 4100450f8fde416ba78f1bdb33a22f16 (127.6.210.131:38743): Couldn't send request to peer 4100450f8fde416ba78f1bdb33a22f16. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 349a01fc3b7243019d0232d51af54366. This is attempt 1: this message will repeat every 5th retry.
I20260924 21:15:51.079777 23684 raft_consensus.cc:1261] T 349a01fc3b7243019d0232d51af54366 P 1eeca863d1c14311add655d8b95dce4b [term 2 FOLLOWER]: Refusing update from remote peer 8e270e70ec6a4095ae7b556b25bebbe4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260924 21:15:51.080353 23871 consensus_queue.cc:1035] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1eeca863d1c14311add655d8b95dce4b" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 42013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260924 21:15:51.081450 23879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:51.081209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.130:39633)
0924 21:15:51.081317 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:51.081360 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":61,"threads_started":1}
I20260924 21:15:51.082345 23883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:51.082141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.129:49497)
0924 21:15:51.082241 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:51.082299 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":41,"threads_started":1}
I20260924 21:15:51.082465 23879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:51.082343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.130:35987)
0924 21:15:51.082393 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:51.082430 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260924 21:15:51.432224 23884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:51.431965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.131:33209)
0924 21:15:51.432104 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:51.432167 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":82,"threads_started":1}
I20260924 21:15:51.552846 23885 ts_tablet_manager.cc:927] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16: Initiating tablet copy from peer 8e270e70ec6a4095ae7b556b25bebbe4 (127.6.210.129:32817)
I20260924 21:15:51.553081 23885 tablet_copy_client.cc:316] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16: tablet copy: Beginning tablet copy session from remote peer at address 127.6.210.129:32817
Sep 24 21:15:51 dist-test-slave-q276 krb5kdc[22290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284549, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for kudu/127.6.210.129@KRBTEST.COM
I20260924 21:15:51.561405 23884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:51.553375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:32817 (local address 127.6.210.131:34231)
0924 21:15:51.553464 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:51.553485 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0924 21:15:51.553573 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:15:51.553808 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:15:51.553812 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:15:51.554117 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:15:51.554185 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:51.554187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:51.555121 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:51.555122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:51.555855 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:15:51.555858 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:15:51.556390 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:15:51.556391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:15:51.556456 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:51.557036 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:15:51.557040 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:15:51.558518 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:15:51.560237 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:15:51.560243 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:15:51.560245 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:15:51.560453 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:15:51.560681 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:15:51.560682 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:15:51.560682 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:15:51.560754 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:15:51.561033 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:15:51.561039 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:15:51.561090 (+ 51us) client_negotiation.cc:769] Sending connection context
0924 21:15:51.561164 (+ 74us) client_negotiation.cc:240] Negotiation successful
0924 21:15:51.561264 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260924 21:15:51.561578 23887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:51.553484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:34231 (local address 127.6.210.129:32817)
0924 21:15:51.553666 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0924 21:15:51.553670 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:15:51.553680 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:15:51.553704 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:15:51.553706 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:15:51.553719 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:15:51.553783 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:15:51.554308 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:51.555006 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:51.555943 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:15:51.556255 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:15:51.556312 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:15:51.558596 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:15:51.558603 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:15:51.558604 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:15:51.558610 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:15:51.560153 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:15:51.560530 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:15:51.560532 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:15:51.560533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:15:51.560590 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:15:51.560825 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:15:51.560825 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:15:51.560825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:15:51.560966 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:15:51.561028 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 21:15:51.561251 (+ 223us) server_negotiation.cc:299] Negotiation successful
0924 21:15:51.561398 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":106,"threads_started":1}
I20260924 21:15:51.562034 23569 tablet_copy_service.cc:140] P 8e270e70ec6a4095ae7b556b25bebbe4: Received BeginTabletCopySession request for tablet 349a01fc3b7243019d0232d51af54366 from peer 4100450f8fde416ba78f1bdb33a22f16 ({username='kudu', principal='kudu/127.6.210.131@KRBTEST.COM'} at 127.6.210.131:34231)
I20260924 21:15:51.562139 23569 tablet_copy_service.cc:161] P 8e270e70ec6a4095ae7b556b25bebbe4: Beginning new tablet copy session on tablet 349a01fc3b7243019d0232d51af54366 from peer 4100450f8fde416ba78f1bdb33a22f16 at {username='kudu', principal='kudu/127.6.210.131@KRBTEST.COM'} at 127.6.210.131:34231: session id = 4100450f8fde416ba78f1bdb33a22f16-349a01fc3b7243019d0232d51af54366
I20260924 21:15:51.562662 23569 tablet_copy_source_session.cc:205] T 349a01fc3b7243019d0232d51af54366 P 8e270e70ec6a4095ae7b556b25bebbe4: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 21:15:51.563663 23885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 349a01fc3b7243019d0232d51af54366. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:15:51.566133 23885 tablet_copy_client.cc:791] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16: tablet copy: Starting download of 0 data blocks...
I20260924 21:15:51.566340 23885 tablet_copy_client.cc:655] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16: tablet copy: Starting download of 1 WAL segments...
I20260924 21:15:51.567201 23885 tablet_copy_client.cc:531] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 21:15:51.568261 23885 tablet_bootstrap.cc:492] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16: Bootstrap starting.
I20260924 21:15:51.569778 23885 log.cc:824] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16: Log is configured to *not* fsync() on all Append() calls
I20260924 21:15:51.572202 23885 tablet_bootstrap.cc:492] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 21:15:51.572433 23885 tablet_bootstrap.cc:492] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16: Bootstrap complete.
I20260924 21:15:51.572585 23885 ts_tablet_manager.cc:1397] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260924 21:15:51.573488 23885 raft_consensus.cc:348] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4100450f8fde416ba78f1bdb33a22f16" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 38743 } } peers { permanent_uuid: "1eeca863d1c14311add655d8b95dce4b" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 42013 } } peers { permanent_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 32817 } }
I20260924 21:15:51.573659 23885 raft_consensus.cc:729] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4100450f8fde416ba78f1bdb33a22f16, State: Initialized, Role: FOLLOWER
I20260924 21:15:51.573796 23885 consensus_queue.cc:260] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4100450f8fde416ba78f1bdb33a22f16" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 38743 } } peers { permanent_uuid: "1eeca863d1c14311add655d8b95dce4b" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 42013 } } peers { permanent_uuid: "8e270e70ec6a4095ae7b556b25bebbe4" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 32817 } }
I20260924 21:15:51.574048 23885 ts_tablet_manager.cc:1428] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16: Time spent starting tablet: real 0.001s user 0.000s sys 0.004s
I20260924 21:15:51.574362 23569 tablet_copy_service.cc:342] P 8e270e70ec6a4095ae7b556b25bebbe4: Request end of tablet copy session 4100450f8fde416ba78f1bdb33a22f16-349a01fc3b7243019d0232d51af54366 received from {username='kudu', principal='kudu/127.6.210.131@KRBTEST.COM'} at 127.6.210.131:34231
I20260924 21:15:51.574442 23569 tablet_copy_service.cc:433] P 8e270e70ec6a4095ae7b556b25bebbe4: ending tablet copy session 4100450f8fde416ba78f1bdb33a22f16-349a01fc3b7243019d0232d51af54366 on tablet 349a01fc3b7243019d0232d51af54366 with peer 4100450f8fde416ba78f1bdb33a22f16
I20260924 21:15:51.574445 23884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:51.574241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.131:46357)
0924 21:15:51.574374 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:51.574406 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260924 21:15:51.574898 23884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:51.574709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.131:56421)
0924 21:15:51.574831 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:51.574863 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260924 21:15:52.083154 23891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:52.082920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.129:40811)
0924 21:15:52.083070 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:52.083110 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":63,"threads_started":1}
I20260924 21:15:52.083170 23890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:52.082920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.130:42139)
0924 21:15:52.083099 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:52.083131 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":104,"threads_started":1}
I20260924 21:15:52.157238 23891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:52.157098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.129:46253)
0924 21:15:52.157166 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:52.157202 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260924 21:15:52.157239 23818 raft_consensus.cc:1203] T 349a01fc3b7243019d0232d51af54366 P 4100450f8fde416ba78f1bdb33a22f16 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260924 21:15:52.157876 23894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:52.157671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.131:48329)
0924 21:15:52.157792 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:52.157834 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":50,"threads_started":1}
I20260924 21:15:53.083953 23895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:53.083696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.130:52289)
0924 21:15:53.083860 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:53.083907 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":65,"threads_started":1}
I20260924 21:15:53.157980 23896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:53.157756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.129:51787)
0924 21:15:53.157893 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:53.157936 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":63,"threads_started":1}
I20260924 21:15:53.158705 23897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:53.158431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.131:44491)
0924 21:15:53.158615 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:53.158664 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":69,"threads_started":1}
I20260924 21:15:54.084792 23898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:54.084533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.130:37195)
0924 21:15:54.084708 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:54.084746 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":84,"threads_started":1}
I20260924 21:15:54.158756 23899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:54.158557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.129:34965)
0924 21:15:54.158681 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:54.158712 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":64,"threads_started":1}
I20260924 21:15:54.159420 23899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:54.159192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.129:60185)
0924 21:15:54.159352 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:54.159383 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260924 21:15:54.159651 23900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:54.159372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.131:57587)
0924 21:15:54.159564 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:54.159605 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":61,"threads_started":1}
I20260924 21:15:55.085621 23901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:55.085395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.130:42035)
0924 21:15:55.085541 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:55.085575 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":67,"threads_started":1}
I20260924 21:15:55.160032 23902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:55.159848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.129:43571)
0924 21:15:55.159963 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:55.159989 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":56,"threads_started":1}
I20260924 21:15:55.160386 23903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:55.160197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.131:47259)
0924 21:15:55.160311 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:55.160342 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":58,"threads_started":1}
I20260924 21:15:56.086614 23905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:56.086363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.130:43737)
0924 21:15:56.086518 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:56.086567 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":68,"threads_started":1}
I20260924 21:15:56.160920 23906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:56.160586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.129:47595)
0924 21:15:56.160823 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:56.160875 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":102,"threads_started":1}
I20260924 21:15:56.161258 23907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:56.160978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.131:50223)
0924 21:15:56.161167 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:56.161203 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":71,"threads_started":1}
I20260924 21:15:57.087488 23908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:57.087242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.130:45917)
0924 21:15:57.087398 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:57.087444 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":102,"threads_started":1}
I20260924 21:15:57.161947 23909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:57.161674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.131:57383)
0924 21:15:57.161846 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:57.161890 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":65,"threads_started":1}
I20260924 21:15:57.161969 23910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:57.161722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.129:35819)
0924 21:15:57.161898 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:57.161929 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":122,"threads_started":1}
I20260924 21:15:58.088392 23911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:58.088135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.130:56319)
0924 21:15:58.088298 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:58.088344 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":73,"threads_started":1}
I20260924 21:15:58.162737 23913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:58.162512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.129:40347)
0924 21:15:58.162651 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:58.162692 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
I20260924 21:15:58.162763 23912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:58.162515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.131:32909)
0924 21:15:58.162677 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:58.162722 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":119,"threads_started":1}
I20260924 21:15:59.089167 23914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:59.088935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.130:47187)
0924 21:15:59.089073 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:59.089121 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":73,"threads_started":1}
I20260924 21:15:59.163488 23915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:59.163252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.129:58133)
0924 21:15:59.163388 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:59.163440 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":58,"threads_started":1}
I20260924 21:15:59.163520 23916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:15:59.163306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.131:38501)
0924 21:15:59.163446 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0924 21:15:59.163480 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":63,"threads_started":1}
I20260924 21:16:00.090097 23917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:00.089864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.130:57889)
0924 21:16:00.090003 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:00.090050 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":71,"threads_started":1}
I20260924 21:16:00.164376 23919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:00.164116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.129:51147)
0924 21:16:00.164292 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:00.164320 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":100,"threads_started":1}
I20260924 21:16:00.164376 23918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:00.164099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.131:43117)
0924 21:16:00.164269 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:00.164321 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":73,"threads_started":1}
I20260924 21:16:01.091041 23920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:01.090819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.130:44453)
0924 21:16:01.090958 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:01.090996 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":68,"threads_started":1}
I20260924 21:16:01.165131 23921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:01.164908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.129:43873)
0924 21:16:01.165040 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:01.165085 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":86,"threads_started":1}
I20260924 21:16:01.165138 23922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:01.164910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.131:44299)
0924 21:16:01.165054 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:01.165095 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":101,"threads_started":1}
I20260924 21:16:02.091873 23923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:02.091577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.130:33589)
0924 21:16:02.091767 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:02.091820 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":82,"threads_started":1}
I20260924 21:16:02.165925 23924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:02.165664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.131:39777)
0924 21:16:02.165806 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:02.165848 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":81,"threads_started":1}
I20260924 21:16:02.165979 23925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:02.165664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.129:39699)
0924 21:16:02.165886 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:02.165935 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":172,"threads_started":1}
I20260924 21:16:03.092716 23926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:03.092456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.130:41679)
0924 21:16:03.092617 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:03.092665 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":66,"threads_started":1}
I20260924 21:16:03.166671 23927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:03.166445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.131:48257)
0924 21:16:03.166587 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:03.166630 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":63,"threads_started":1}
I20260924 21:16:03.166667 23928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:03.166451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.129:57993)
0924 21:16:03.166586 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:03.166622 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":85,"threads_started":1}
I20260924 21:16:04.093597 23929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:04.093356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.130:35135)
0924 21:16:04.093509 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:04.093549 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":63,"threads_started":1}
I20260924 21:16:04.167351 23930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:04.167124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.131:33437)
0924 21:16:04.167262 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:04.167305 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":71,"threads_started":1}
I20260924 21:16:04.167475 23931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:04.167217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36237 (local address 127.6.210.129:36679)
0924 21:16:04.167333 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:04.167391 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.190:36237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":72,"threads_started":1}
I20260924 21:16:04.437103 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 23431
I20260924 21:16:04.441865 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 23599
I20260924 21:16:04.446333 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 23734
2026-09-24T21:16:04Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58569 ms)
[----------] 30 tests from SecurityITest (231292 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:16:04 dist-test-slave-q276 krb5kdc[23934](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:16:04 dist-test-slave-q276 krb5kdc[23934](info): set up 2 sockets
Sep 24 21:16:04 dist-test-slave-q276 krb5kdc[23934](info): commencing operation
krb5kdc: starting...
W20260924 21:16:05.497985 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:16:05 dist-test-slave-q276 krb5kdc[23934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:16:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:16:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.210.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
I20260924 21:16:05.626055 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:37823
--webserver_interface=127.6.210.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:36837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:37823
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:05.708431 23947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:05.708650 23947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:05.708673 23947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:05.709836 23947 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:05.709883 23947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:05.709898 23947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:05.709911 23947 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:05.709923 23947 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:05.711516 23947 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:36837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:37823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:37823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:05.711771 23947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:05.712013 23947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:05.717187 23955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:05.717191 23952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:05.717370 23947 server_base.cc:1034] running on GCE node
W20260924 21:16:05.717374 23953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:05.717734 23947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:05.718075 23947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:05.719249 23947 hybrid_clock.cc:648] HybridClock initialized: now 1790284565719226 us; error 65 us; skew 500 ppm
Sep 24 21:16:05 dist-test-slave-q276 krb5kdc[23934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284565, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:05.721796 23947 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:16:05.722342 23947 webserver.cc:460] Webserver started at http://127.6.210.190:44323/ using document root <none> and password file <none>
I20260924 21:16:05.722589 23947 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:05.722657 23947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:05.722800 23947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:05.723850 23947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "a774b6f451174827aec721486c2be0f1"
format_stamp: "Formatted at 2026-09-24 21:16:05 on dist-test-slave-q276"
server_key: "42f5688d9e297f3acd13f053c3131710"
server_key_iv: "7a1210a1e2f5053dec172dd05642c4c7"
server_key_version: "encryptionkey@0"
I20260924 21:16:05.724223 23947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "a774b6f451174827aec721486c2be0f1"
format_stamp: "Formatted at 2026-09-24 21:16:05 on dist-test-slave-q276"
server_key: "42f5688d9e297f3acd13f053c3131710"
server_key_iv: "7a1210a1e2f5053dec172dd05642c4c7"
server_key_version: "encryptionkey@0"
I20260924 21:16:05.725567 23947 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:16:05.726444 23962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:05.726661 23947 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:16:05.726764 23947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "a774b6f451174827aec721486c2be0f1"
format_stamp: "Formatted at 2026-09-24 21:16:05 on dist-test-slave-q276"
server_key: "42f5688d9e297f3acd13f053c3131710"
server_key_iv: "7a1210a1e2f5053dec172dd05642c4c7"
server_key_version: "encryptionkey@0"
I20260924 21:16:05.726848 23947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:05.735046 23947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:05.736869 23947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:05.737035 23947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:05.741166 23947 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:37823
I20260924 21:16:05.741197 24014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:37823 every 8 connection(s)
I20260924 21:16:05.741560 23947 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:16:05.742486 24015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:05.744711 24015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a774b6f451174827aec721486c2be0f1: Bootstrap starting.
I20260924 21:16:05.745285 24015 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a774b6f451174827aec721486c2be0f1: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:05.745564 24015 log.cc:824] T 00000000000000000000000000000000 P a774b6f451174827aec721486c2be0f1: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:05.746243 24015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a774b6f451174827aec721486c2be0f1: No bootstrap required, opened a new log
I20260924 21:16:05.747834 24015 raft_consensus.cc:348] T 00000000000000000000000000000000 P a774b6f451174827aec721486c2be0f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a774b6f451174827aec721486c2be0f1" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37823 } }
I20260924 21:16:05.748011 24015 raft_consensus.cc:374] T 00000000000000000000000000000000 P a774b6f451174827aec721486c2be0f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:05.748075 24015 raft_consensus.cc:729] T 00000000000000000000000000000000 P a774b6f451174827aec721486c2be0f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a774b6f451174827aec721486c2be0f1, State: Initialized, Role: FOLLOWER
I20260924 21:16:05.748224 24015 consensus_queue.cc:260] T 00000000000000000000000000000000 P a774b6f451174827aec721486c2be0f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a774b6f451174827aec721486c2be0f1" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37823 } }
I20260924 21:16:05.748284 24015 raft_consensus.cc:388] T 00000000000000000000000000000000 P a774b6f451174827aec721486c2be0f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:16:05.748342 24015 raft_consensus.cc:482] T 00000000000000000000000000000000 P a774b6f451174827aec721486c2be0f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:16:05.748389 24015 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a774b6f451174827aec721486c2be0f1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:05.748904 24015 raft_consensus.cc:504] T 00000000000000000000000000000000 P a774b6f451174827aec721486c2be0f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a774b6f451174827aec721486c2be0f1" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37823 } }
I20260924 21:16:05.749023 24015 leader_election.cc:302] T 00000000000000000000000000000000 P a774b6f451174827aec721486c2be0f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a774b6f451174827aec721486c2be0f1; no voters:
I20260924 21:16:05.749199 24015 leader_election.cc:288] T 00000000000000000000000000000000 P a774b6f451174827aec721486c2be0f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:16:05.749219 24018 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a774b6f451174827aec721486c2be0f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:05.749368 24018 raft_consensus.cc:686] T 00000000000000000000000000000000 P a774b6f451174827aec721486c2be0f1 [term 1 LEADER]: Becoming Leader. State: Replica: a774b6f451174827aec721486c2be0f1, State: Running, Role: LEADER
I20260924 21:16:05.749447 24015 sys_catalog.cc:564] T 00000000000000000000000000000000 P a774b6f451174827aec721486c2be0f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:05.749552 24018 consensus_queue.cc:237] T 00000000000000000000000000000000 P a774b6f451174827aec721486c2be0f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a774b6f451174827aec721486c2be0f1" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37823 } }
I20260924 21:16:05.749924 24020 sys_catalog.cc:455] T 00000000000000000000000000000000 P a774b6f451174827aec721486c2be0f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a774b6f451174827aec721486c2be0f1. Latest consensus state: current_term: 1 leader_uuid: "a774b6f451174827aec721486c2be0f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a774b6f451174827aec721486c2be0f1" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37823 } } }
I20260924 21:16:05.750099 24019 sys_catalog.cc:455] T 00000000000000000000000000000000 P a774b6f451174827aec721486c2be0f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a774b6f451174827aec721486c2be0f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a774b6f451174827aec721486c2be0f1" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37823 } } }
I20260924 21:16:05.750180 24020 sys_catalog.cc:458] T 00000000000000000000000000000000 P a774b6f451174827aec721486c2be0f1 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:05.750242 24019 sys_catalog.cc:458] T 00000000000000000000000000000000 P a774b6f451174827aec721486c2be0f1 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:05.750499 24027 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:16:05.751017 24027 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:16:05.751096 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 23947
I20260924 21:16:05.751209 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:16:05.751415 6986 external_mini_cluster.cc:1428] Setting key 68df42a7b4035510e739da79e9393d3a
I20260924 21:16:05.752502 24027 catalog_manager.cc:1347] Generated new cluster ID: c023d17250c44bc386f0aaa113ddb843
I20260924 21:16:05.752570 24027 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 21:16:05 dist-test-slave-q276 krb5kdc[23934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:16:05.763422 24038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:05.752338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59768 (local address 127.6.210.190:37823)
0924 21:16:05.752574 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:05.752582 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:05.752611 (+ 29us) server_negotiation.cc:407] Connection header received
0924 21:16:05.752719 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:05.752726 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:05.752779 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:05.752920 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:05.753610 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:05.755113 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:05.756120 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:05.756746 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:05.756856 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:05.759699 (+ 2843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:05.759703 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:05.759703 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:05.759715 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:05.761691 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:05.762120 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:05.762123 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:05.762123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:05.762227 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:05.762465 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:05.762465 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:05.762465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:05.762767 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:05.762915 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:05.763038 (+ 123us) server_negotiation.cc:299] Negotiation successful
0924 21:16:05.763169 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":80,"threads_started":1}
I20260924 21:16:05.773114 24027 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:16:05.773837 24027 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:16:05.783947 24027 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a774b6f451174827aec721486c2be0f1: Generated new TSK 0
I20260924 21:16:05.784173 24027 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.210.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
I20260924 21:16:05.814704 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.129:0
--local_ip_for_outbound_sockets=127.6.210.129
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:37823
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:36837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:05.925239 24041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:05.925479 24041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:05.925508 24041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:05.926910 24041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:05.926964 24041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:05.926996 24041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.129
I20260924 21:16:05.928654 24041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:36837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:37823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.129
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:05.928928 24041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:05.929253 24041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:05.935025 24046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:05.935037 24049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:05.935143 24041 server_base.cc:1034] running on GCE node
W20260924 21:16:05.935129 24047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:05.935498 24041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:05.935720 24041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:05.936903 24041 hybrid_clock.cc:648] HybridClock initialized: now 1790284565936868 us; error 31 us; skew 500 ppm
Sep 24 21:16:05 dist-test-slave-q276 krb5kdc[23934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284565, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:05.939703 24041 init.cc:377] Logged in from keytab as kudu/127.6.210.129@KRBTEST.COM (short username kudu)
I20260924 21:16:05.940163 24041 webserver.cc:460] Webserver started at http://127.6.210.129:46781/ using document root <none> and password file <none>
I20260924 21:16:05.940363 24041 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:05.940404 24041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:05.940496 24041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:05.941546 24041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-0/data/instance:
uuid: "615975d774ee48f1a56e2286986768b7"
format_stamp: "Formatted at 2026-09-24 21:16:05 on dist-test-slave-q276"
server_key: "37f81ed9ccbc7a960d771ba5ea23a354"
server_key_iv: "c97bc55d39d2165f7ad1c3902c26c724"
server_key_version: "encryptionkey@0"
I20260924 21:16:05.941848 24041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance:
uuid: "615975d774ee48f1a56e2286986768b7"
format_stamp: "Formatted at 2026-09-24 21:16:05 on dist-test-slave-q276"
server_key: "37f81ed9ccbc7a960d771ba5ea23a354"
server_key_iv: "c97bc55d39d2165f7ad1c3902c26c724"
server_key_version: "encryptionkey@0"
I20260924 21:16:05.943241 24041 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:16:05.944067 24056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:05.944384 24041 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:16:05.944460 24041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-0/wal
uuid: "615975d774ee48f1a56e2286986768b7"
format_stamp: "Formatted at 2026-09-24 21:16:05 on dist-test-slave-q276"
server_key: "37f81ed9ccbc7a960d771ba5ea23a354"
server_key_iv: "c97bc55d39d2165f7ad1c3902c26c724"
server_key_version: "encryptionkey@0"
I20260924 21:16:05.944576 24041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:05.965550 24041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:05.968017 24041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:05.968170 24041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:05.968429 24041 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:16:05.968746 24041 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:16:05.968780 24041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:05.968803 24041 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:16:05.968816 24041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:05.974582 24041 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.129:44537
I20260924 21:16:05.974630 24169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.129:44537 every 8 connection(s)
I20260924 21:16:05.974996 24041 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
I20260924 21:16:05.980682 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 24041
I20260924 21:16:05.980803 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance
I20260924 21:16:05.980984 6986 external_mini_cluster.cc:1428] Setting key 1dd234f3e69650bc275d318fc009897e
Sep 24 21:16:05 dist-test-slave-q276 krb5kdc[23934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284565, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:16:05.985397 24038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:05.975767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:36873 (local address 127.6.210.190:37823)
0924 21:16:05.975859 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:05.975862 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:05.976048 (+ 186us) server_negotiation.cc:407] Connection header received
0924 21:16:05.976643 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:05.976645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:05.976661 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:05.976714 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:05.977769 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:05.978409 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:05.979520 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:05.979752 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:05.979845 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:05.982252 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:05.982253 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:05.982253 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:05.982280 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:05.983828 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:05.984411 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:05.984412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:05.984413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:05.984472 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:05.984755 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:05.984755 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:05.984755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:05.984859 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:05.984913 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:05.985257 (+ 344us) server_negotiation.cc:299] Negotiation successful
0924 21:16:05.985330 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260924 21:16:05.985666 24172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:05.975724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37823 (local address 127.6.210.129:36873)
0924 21:16:05.975908 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:05.975957 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:05.976520 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:05.976810 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:05.976814 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:05.977448 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:05.977654 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:05.977660 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:05.978510 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:05.978512 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:05.979390 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:05.979392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:05.979857 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:05.979858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:05.979994 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:05.980434 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:05.980461 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:05.982137 (+ 1676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:05.983986 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:05.983992 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:05.983995 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:05.984330 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:05.984544 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:05.984545 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:05.984545 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:05.984688 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:05.984926 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:05.984933 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:05.985099 (+ 166us) client_negotiation.cc:769] Sending connection context
0924 21:16:05.985226 (+ 127us) client_negotiation.cc:240] Negotiation successful
0924 21:16:05.985412 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":84,"threads_started":1}
I20260924 21:16:05.986274 24170 heartbeater.cc:347] Connected to a master server at 127.6.210.190:37823
I20260924 21:16:05.986402 24170 heartbeater.cc:464] Registering TS with master...
I20260924 21:16:05.986639 24170 heartbeater.cc:511] Master 127.6.210.190:37823 requested a full tablet report, sending...
I20260924 21:16:05.987174 23979 ts_manager.cc:194] Registered new tserver with Master: 615975d774ee48f1a56e2286986768b7 (127.6.210.129:44537)
I20260924 21:16:05.987972 23979 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.129@KRBTEST.COM'} at 127.6.210.129:36873
WARNING: no policy specified for kudu/127.6.210.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
I20260924 21:16:06.007311 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.130:0
--local_ip_for_outbound_sockets=127.6.210.130
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:37823
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:36837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:06.091200 24175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:06.091470 24175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:06.091512 24175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:06.092711 24175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:06.092790 24175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:06.092834 24175 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.130
I20260924 21:16:06.094321 24175 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:36837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:37823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.130
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:06.094599 24175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:06.094841 24175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:06.100075 24183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:06.100164 24175 server_base.cc:1034] running on GCE node
W20260924 21:16:06.100044 24181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:06.100203 24180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:06.100490 24175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:06.100800 24175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:06.101977 24175 hybrid_clock.cc:648] HybridClock initialized: now 1790284566101958 us; error 48 us; skew 500 ppm
Sep 24 21:16:06 dist-test-slave-q276 krb5kdc[23934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284566, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:06.104496 24175 init.cc:377] Logged in from keytab as kudu/127.6.210.130@KRBTEST.COM (short username kudu)
I20260924 21:16:06.104993 24175 webserver.cc:460] Webserver started at http://127.6.210.130:44261/ using document root <none> and password file <none>
I20260924 21:16:06.105244 24175 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:06.105317 24175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:06.105459 24175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:06.106643 24175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-1/data/instance:
uuid: "40a0ebe1286745c08a31fe5a86b2dc26"
format_stamp: "Formatted at 2026-09-24 21:16:06 on dist-test-slave-q276"
server_key: "df73f654fa54bd1090c38e0f54bb4367"
server_key_iv: "33c28e87a6dc5ddffaa8df58ac193d2d"
server_key_version: "encryptionkey@0"
I20260924 21:16:06.107023 24175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance:
uuid: "40a0ebe1286745c08a31fe5a86b2dc26"
format_stamp: "Formatted at 2026-09-24 21:16:06 on dist-test-slave-q276"
server_key: "df73f654fa54bd1090c38e0f54bb4367"
server_key_iv: "33c28e87a6dc5ddffaa8df58ac193d2d"
server_key_version: "encryptionkey@0"
I20260924 21:16:06.108381 24175 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:16:06.109277 24190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:06.109505 24175 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:06.109628 24175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-1/wal
uuid: "40a0ebe1286745c08a31fe5a86b2dc26"
format_stamp: "Formatted at 2026-09-24 21:16:06 on dist-test-slave-q276"
server_key: "df73f654fa54bd1090c38e0f54bb4367"
server_key_iv: "33c28e87a6dc5ddffaa8df58ac193d2d"
server_key_version: "encryptionkey@0"
I20260924 21:16:06.109720 24175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:06.130569 24175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:06.132797 24175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:06.132992 24175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:06.133250 24175 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:16:06.133608 24175 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:16:06.133672 24175 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:06.133721 24175 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:16:06.133749 24175 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:06.140400 24175 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.130:35491
I20260924 21:16:06.140499 24303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.130:35491 every 8 connection(s)
I20260924 21:16:06.140838 24175 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
I20260924 21:16:06.142350 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 24175
I20260924 21:16:06.142447 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance
I20260924 21:16:06.142655 6986 external_mini_cluster.cc:1428] Setting key f559dc7ed07e973abae9a4257e91694d
Sep 24 21:16:06 dist-test-slave-q276 krb5kdc[23934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284566, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:16:06.151706 24038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:06.141623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:52219 (local address 127.6.210.190:37823)
0924 21:16:06.141703 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:06.141706 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:06.141940 (+ 234us) server_negotiation.cc:407] Connection header received
0924 21:16:06.142629 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:06.142630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:06.142642 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:06.142740 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:06.144081 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.144575 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.145704 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.146030 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.146101 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:06.148385 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:06.148386 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:06.148386 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:06.148396 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:06.150304 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:06.150747 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:06.150749 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:06.150749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:06.150808 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:06.151038 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:06.151039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:06.151039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:06.151186 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:06.151267 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:06.151521 (+ 254us) server_negotiation.cc:299] Negotiation successful
0924 21:16:06.151614 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260924 21:16:06.151789 24306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:06.141607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37823 (local address 127.6.210.130:52219)
0924 21:16:06.141791 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:06.141832 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:06.142509 (+ 677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:06.142861 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:06.142865 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:06.143656 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:06.143945 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:06.143952 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.144674 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.144675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:06.145593 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:06.145595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.146120 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.146121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:06.146240 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:06.146864 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:06.146872 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:06.148292 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:06.150380 (+ 2088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:06.150386 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:06.150388 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:06.150675 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:06.150875 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:06.150876 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:06.150876 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:06.150971 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:06.151262 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:06.151267 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:06.151385 (+ 118us) client_negotiation.cc:769] Sending connection context
0924 21:16:06.151472 (+ 87us) client_negotiation.cc:240] Negotiation successful
0924 21:16:06.151597 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":110,"threads_started":1}
WARNING: no policy specified for kudu/127.6.210.131@KRBTEST.COM; defaulting to no policy
I20260924 21:16:06.152307 24304 heartbeater.cc:347] Connected to a master server at 127.6.210.190:37823
I20260924 21:16:06.152392 24304 heartbeater.cc:464] Registering TS with master...
I20260924 21:16:06.152544 24304 heartbeater.cc:511] Master 127.6.210.190:37823 requested a full tablet report, sending...
I20260924 21:16:06.152930 23979 ts_manager.cc:194] Registered new tserver with Master: 40a0ebe1286745c08a31fe5a86b2dc26 (127.6.210.130:35491)
I20260924 21:16:06.153314 23979 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.130@KRBTEST.COM'} at 127.6.210.130:52219
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
I20260924 21:16:06.169414 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.131:0
--local_ip_for_outbound_sockets=127.6.210.131
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:37823
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:36837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:06.252328 24309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:06.252554 24309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:06.252581 24309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:06.253664 24309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:06.253736 24309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:06.253790 24309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.131
I20260924 21:16:06.255302 24309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:36837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:37823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.131
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:06.255504 24309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:06.255677 24309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:06.260998 24314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:06.261050 24309 server_base.cc:1034] running on GCE node
W20260924 21:16:06.261157 24315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:06.261228 24317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:06.261483 24309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:06.261749 24309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:06.262894 24309 hybrid_clock.cc:648] HybridClock initialized: now 1790284566262850 us; error 61 us; skew 500 ppm
Sep 24 21:16:06 dist-test-slave-q276 krb5kdc[23934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284566, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:06.265594 24309 init.cc:377] Logged in from keytab as kudu/127.6.210.131@KRBTEST.COM (short username kudu)
I20260924 21:16:06.266043 24309 webserver.cc:460] Webserver started at http://127.6.210.131:37463/ using document root <none> and password file <none>
I20260924 21:16:06.266312 24309 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:06.266377 24309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:06.266515 24309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:06.267594 24309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-2/data/instance:
uuid: "623a14d221f1410aa430e37f39a36e56"
format_stamp: "Formatted at 2026-09-24 21:16:06 on dist-test-slave-q276"
server_key: "e635f4cf00d3ca8541cead7e5607e0cf"
server_key_iv: "5472ebb58953eb2e164af9f72bc04a8a"
server_key_version: "encryptionkey@0"
I20260924 21:16:06.267997 24309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance:
uuid: "623a14d221f1410aa430e37f39a36e56"
format_stamp: "Formatted at 2026-09-24 21:16:06 on dist-test-slave-q276"
server_key: "e635f4cf00d3ca8541cead7e5607e0cf"
server_key_iv: "5472ebb58953eb2e164af9f72bc04a8a"
server_key_version: "encryptionkey@0"
I20260924 21:16:06.269356 24309 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:16:06.270325 24324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:06.270550 24309 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:06.270654 24309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-2/wal
uuid: "623a14d221f1410aa430e37f39a36e56"
format_stamp: "Formatted at 2026-09-24 21:16:06 on dist-test-slave-q276"
server_key: "e635f4cf00d3ca8541cead7e5607e0cf"
server_key_iv: "5472ebb58953eb2e164af9f72bc04a8a"
server_key_version: "encryptionkey@0"
I20260924 21:16:06.270742 24309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:06.295274 24309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:06.297717 24309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:06.297976 24309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:06.298228 24309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:16:06.298578 24309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:16:06.298647 24309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:06.298689 24309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:16:06.298717 24309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:06.304666 24309 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.131:33639
I20260924 21:16:06.304714 24437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.131:33639 every 8 connection(s)
I20260924 21:16:06.305063 24309 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
Sep 24 21:16:06 dist-test-slave-q276 krb5kdc[23934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284566, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:16:06.314388 24038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:06.305820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:35195 (local address 127.6.210.190:37823)
0924 21:16:06.305894 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:06.305897 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:06.306132 (+ 235us) server_negotiation.cc:407] Connection header received
0924 21:16:06.306646 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:06.306647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:06.306658 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:06.306713 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:06.307713 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.308213 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.309182 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.309366 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.309424 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:06.311522 (+ 2098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:06.311523 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:06.311523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:06.311530 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:06.312930 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:06.313417 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:06.313418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:06.313418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:06.313472 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:06.313770 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:06.313771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:06.313771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:06.313885 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:06.313940 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:06.314206 (+ 266us) server_negotiation.cc:299] Negotiation successful
0924 21:16:06.314322 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260924 21:16:06.314698 24440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:06.305811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37823 (local address 127.6.210.131:35195)
0924 21:16:06.306004 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:06.306039 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:06.306557 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:06.306767 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:06.306771 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:06.307388 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:06.307583 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:06.307587 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.308297 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.308298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:06.309095 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:06.309097 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.309451 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.309452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:06.309568 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:06.310038 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:06.310046 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:06.311426 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:06.313013 (+ 1587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:06.313019 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:06.313021 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:06.313327 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:06.313541 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:06.313541 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:06.313542 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:06.313693 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:06.313951 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:06.313956 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:06.314082 (+ 126us) client_negotiation.cc:769] Sending connection context
0924 21:16:06.314176 (+ 94us) client_negotiation.cc:240] Negotiation successful
0924 21:16:06.314371 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":119,"threads_started":1}
I20260924 21:16:06.314806 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 24309
I20260924 21:16:06.314898 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance
I20260924 21:16:06.315150 6986 external_mini_cluster.cc:1428] Setting key cc1fdee52af9e0af6be487547c2dcae5
I20260924 21:16:06.315301 24438 heartbeater.cc:347] Connected to a master server at 127.6.210.190:37823
I20260924 21:16:06.315412 24438 heartbeater.cc:464] Registering TS with master...
I20260924 21:16:06.315654 24438 heartbeater.cc:511] Master 127.6.210.190:37823 requested a full tablet report, sending...
I20260924 21:16:06.316212 23979 ts_manager.cc:194] Registered new tserver with Master: 623a14d221f1410aa430e37f39a36e56 (127.6.210.131:33639)
I20260924 21:16:06.316676 23979 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.131@KRBTEST.COM'} at 127.6.210.131:35195
I20260924 21:16:06.327189 6986 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:16:06.334841 24038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:06.328625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35344 (local address 127.6.210.190:37823)
0924 21:16:06.328685 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:06.328687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:06.328799 (+ 112us) server_negotiation.cc:407] Connection header received
0924 21:16:06.328880 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:06.328880 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:06.328891 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:06.328947 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:06.329504 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.330087 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.330888 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.331094 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.331177 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:06.332393 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:06.332395 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:06.332395 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:06.332404 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:06.333653 (+ 1249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:06.334003 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:06.334005 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:06.334005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:06.334063 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:06.334259 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:06.334286 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:06.334286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:06.334440 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:06.334555 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:06.334662 (+ 107us) server_negotiation.cc:299] Negotiation successful
0924 21:16:06.334740 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260924 21:16:06.336634 23979 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35344:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:16:06.338238 23979 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:16:06.345250 24452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:06.340857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47638 (local address 127.6.210.130:35491)
0924 21:16:06.341053 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:06.341058 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:06.341069 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:16:06.341116 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:06.341117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:06.341129 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:06.341228 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:06.341455 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.342500 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.344082 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.344809 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.344883 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:06.344940 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:06.345110 (+ 170us) server_negotiation.cc:299] Negotiation successful
0924 21:16:06.345163 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":103,"threads_started":1}
I20260924 21:16:06.345244 24451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:06.340640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:35491 (local address 127.0.0.1:47638)
0924 21:16:06.340838 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:06.340945 (+ 107us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:06.341017 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:06.341234 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:06.341238 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:06.341241 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:06.341312 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:06.341314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.342602 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.342603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:06.343981 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:06.343984 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.344909 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.344910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:06.344989 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:06.345002 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 21:16:06.345084 (+ 82us) client_negotiation.cc:240] Negotiation successful
0924 21:16:06.345155 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":86,"threads_started":1}
I20260924 21:16:06.345458 24449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:06.340645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:44537 (local address 127.0.0.1:58046)
0924 21:16:06.341626 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:06.341636 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:06.341697 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:06.341861 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:06.341861 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:06.341863 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:06.341908 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:06.341910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.343029 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.343030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:06.344218 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:06.344220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.345057 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.345058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:06.345114 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:06.345118 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:16:06.345169 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 21:16:06.345202 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":967,"mutex_wait_us":4,"thread_start_us":100,"threads_started":1}
I20260924 21:16:06.345690 24453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:06.340878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58046 (local address 127.6.210.129:44537)
0924 21:16:06.341385 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:06.341390 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:06.341690 (+ 300us) server_negotiation.cc:407] Connection header received
0924 21:16:06.341768 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:06.341770 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:06.341782 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:06.341841 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:06.341976 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.342917 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.344340 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.344881 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.344960 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:06.345021 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:06.345290 (+ 269us) server_negotiation.cc:299] Negotiation successful
0924 21:16:06.345345 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":104,"threads_started":1}
I20260924 21:16:06.345784 24454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:06.341100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41840 (local address 127.6.210.131:33639)
0924 21:16:06.341324 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:06.341328 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:06.341336 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:16:06.341421 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:06.341424 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:06.341443 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:06.341530 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:06.342495 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.343231 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.344379 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.345101 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.345167 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:06.345216 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:06.345658 (+ 442us) server_negotiation.cc:299] Negotiation successful
0924 21:16:06.345709 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":97,"threads_started":1}
I20260924 21:16:06.345754 24450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:06.340640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:33639 (local address 127.0.0.1:41840)
0924 21:16:06.341143 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:06.341159 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:06.341204 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:06.342341 (+ 1137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:06.342342 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:06.342344 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:06.342388 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:06.342390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.343360 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.343362 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:06.344255 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:06.344258 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.345251 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.345252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:06.345321 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:06.345325 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:16:06.345643 (+ 318us) client_negotiation.cc:240] Negotiation successful
0924 21:16:06.345688 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"mutex_wait_us":2,"thread_start_us":73,"threads_started":1}
I20260924 21:16:06.346189 24238 tablet_service.cc:1467] Processing CreateTablet for tablet cfb7748b170f4ee1a8b91f2a40b80db3 (DEFAULT_TABLE table=test-table [id=7fe5712b8ef34e959ae486b0c2ba2b6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:06.346603 24238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfb7748b170f4ee1a8b91f2a40b80db3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:06.346534 24104 tablet_service.cc:1467] Processing CreateTablet for tablet cfb7748b170f4ee1a8b91f2a40b80db3 (DEFAULT_TABLE table=test-table [id=7fe5712b8ef34e959ae486b0c2ba2b6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:06.346652 24372 tablet_service.cc:1467] Processing CreateTablet for tablet cfb7748b170f4ee1a8b91f2a40b80db3 (DEFAULT_TABLE table=test-table [id=7fe5712b8ef34e959ae486b0c2ba2b6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:06.346800 24104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfb7748b170f4ee1a8b91f2a40b80db3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:06.346948 24372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfb7748b170f4ee1a8b91f2a40b80db3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:06.349472 24455 tablet_bootstrap.cc:492] T cfb7748b170f4ee1a8b91f2a40b80db3 P 615975d774ee48f1a56e2286986768b7: Bootstrap starting.
I20260924 21:16:06.350310 24455 tablet_bootstrap.cc:654] T cfb7748b170f4ee1a8b91f2a40b80db3 P 615975d774ee48f1a56e2286986768b7: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:06.350354 24456 tablet_bootstrap.cc:492] T cfb7748b170f4ee1a8b91f2a40b80db3 P 623a14d221f1410aa430e37f39a36e56: Bootstrap starting.
I20260924 21:16:06.350574 24457 tablet_bootstrap.cc:492] T cfb7748b170f4ee1a8b91f2a40b80db3 P 40a0ebe1286745c08a31fe5a86b2dc26: Bootstrap starting.
I20260924 21:16:06.350931 24455 log.cc:824] T cfb7748b170f4ee1a8b91f2a40b80db3 P 615975d774ee48f1a56e2286986768b7: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:06.351241 24457 tablet_bootstrap.cc:654] T cfb7748b170f4ee1a8b91f2a40b80db3 P 40a0ebe1286745c08a31fe5a86b2dc26: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:06.351751 24455 tablet_bootstrap.cc:492] T cfb7748b170f4ee1a8b91f2a40b80db3 P 615975d774ee48f1a56e2286986768b7: No bootstrap required, opened a new log
I20260924 21:16:06.351861 24455 ts_tablet_manager.cc:1397] T cfb7748b170f4ee1a8b91f2a40b80db3 P 615975d774ee48f1a56e2286986768b7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260924 21:16:06.351877 24457 log.cc:824] T cfb7748b170f4ee1a8b91f2a40b80db3 P 40a0ebe1286745c08a31fe5a86b2dc26: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:06.352651 24457 tablet_bootstrap.cc:492] T cfb7748b170f4ee1a8b91f2a40b80db3 P 40a0ebe1286745c08a31fe5a86b2dc26: No bootstrap required, opened a new log
I20260924 21:16:06.352726 24457 ts_tablet_manager.cc:1397] T cfb7748b170f4ee1a8b91f2a40b80db3 P 40a0ebe1286745c08a31fe5a86b2dc26: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:16:06.352687 24456 tablet_bootstrap.cc:654] T cfb7748b170f4ee1a8b91f2a40b80db3 P 623a14d221f1410aa430e37f39a36e56: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:06.353186 24456 log.cc:824] T cfb7748b170f4ee1a8b91f2a40b80db3 P 623a14d221f1410aa430e37f39a36e56: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:06.353878 24456 tablet_bootstrap.cc:492] T cfb7748b170f4ee1a8b91f2a40b80db3 P 623a14d221f1410aa430e37f39a36e56: No bootstrap required, opened a new log
I20260924 21:16:06.353758 24455 raft_consensus.cc:348] T cfb7748b170f4ee1a8b91f2a40b80db3 P 615975d774ee48f1a56e2286986768b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623a14d221f1410aa430e37f39a36e56" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 33639 } } peers { permanent_uuid: "40a0ebe1286745c08a31fe5a86b2dc26" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 35491 } } peers { permanent_uuid: "615975d774ee48f1a56e2286986768b7" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 44537 } }
I20260924 21:16:06.353940 24455 raft_consensus.cc:374] T cfb7748b170f4ee1a8b91f2a40b80db3 P 615975d774ee48f1a56e2286986768b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:06.353966 24456 ts_tablet_manager.cc:1397] T cfb7748b170f4ee1a8b91f2a40b80db3 P 623a14d221f1410aa430e37f39a36e56: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260924 21:16:06.354017 24455 raft_consensus.cc:729] T cfb7748b170f4ee1a8b91f2a40b80db3 P 615975d774ee48f1a56e2286986768b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 615975d774ee48f1a56e2286986768b7, State: Initialized, Role: FOLLOWER
I20260924 21:16:06.354154 24455 consensus_queue.cc:260] T cfb7748b170f4ee1a8b91f2a40b80db3 P 615975d774ee48f1a56e2286986768b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623a14d221f1410aa430e37f39a36e56" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 33639 } } peers { permanent_uuid: "40a0ebe1286745c08a31fe5a86b2dc26" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 35491 } } peers { permanent_uuid: "615975d774ee48f1a56e2286986768b7" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 44537 } }
I20260924 21:16:06.354439 24455 ts_tablet_manager.cc:1428] T cfb7748b170f4ee1a8b91f2a40b80db3 P 615975d774ee48f1a56e2286986768b7: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260924 21:16:06.354612 24170 heartbeater.cc:503] Master 127.6.210.190:37823 was elected leader, sending a full tablet report...
I20260924 21:16:06.354737 24457 raft_consensus.cc:348] T cfb7748b170f4ee1a8b91f2a40b80db3 P 40a0ebe1286745c08a31fe5a86b2dc26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623a14d221f1410aa430e37f39a36e56" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 33639 } } peers { permanent_uuid: "40a0ebe1286745c08a31fe5a86b2dc26" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 35491 } } peers { permanent_uuid: "615975d774ee48f1a56e2286986768b7" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 44537 } }
I20260924 21:16:06.354879 24457 raft_consensus.cc:374] T cfb7748b170f4ee1a8b91f2a40b80db3 P 40a0ebe1286745c08a31fe5a86b2dc26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:06.354969 24457 raft_consensus.cc:729] T cfb7748b170f4ee1a8b91f2a40b80db3 P 40a0ebe1286745c08a31fe5a86b2dc26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40a0ebe1286745c08a31fe5a86b2dc26, State: Initialized, Role: FOLLOWER
I20260924 21:16:06.355101 24457 consensus_queue.cc:260] T cfb7748b170f4ee1a8b91f2a40b80db3 P 40a0ebe1286745c08a31fe5a86b2dc26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623a14d221f1410aa430e37f39a36e56" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 33639 } } peers { permanent_uuid: "40a0ebe1286745c08a31fe5a86b2dc26" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 35491 } } peers { permanent_uuid: "615975d774ee48f1a56e2286986768b7" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 44537 } }
I20260924 21:16:06.355409 24457 ts_tablet_manager.cc:1428] T cfb7748b170f4ee1a8b91f2a40b80db3 P 40a0ebe1286745c08a31fe5a86b2dc26: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:16:06.355569 24304 heartbeater.cc:503] Master 127.6.210.190:37823 was elected leader, sending a full tablet report...
I20260924 21:16:06.356096 24456 raft_consensus.cc:348] T cfb7748b170f4ee1a8b91f2a40b80db3 P 623a14d221f1410aa430e37f39a36e56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623a14d221f1410aa430e37f39a36e56" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 33639 } } peers { permanent_uuid: "40a0ebe1286745c08a31fe5a86b2dc26" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 35491 } } peers { permanent_uuid: "615975d774ee48f1a56e2286986768b7" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 44537 } }
I20260924 21:16:06.356273 24456 raft_consensus.cc:374] T cfb7748b170f4ee1a8b91f2a40b80db3 P 623a14d221f1410aa430e37f39a36e56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:06.356326 24456 raft_consensus.cc:729] T cfb7748b170f4ee1a8b91f2a40b80db3 P 623a14d221f1410aa430e37f39a36e56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 623a14d221f1410aa430e37f39a36e56, State: Initialized, Role: FOLLOWER
I20260924 21:16:06.356437 24456 consensus_queue.cc:260] T cfb7748b170f4ee1a8b91f2a40b80db3 P 623a14d221f1410aa430e37f39a36e56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623a14d221f1410aa430e37f39a36e56" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 33639 } } peers { permanent_uuid: "40a0ebe1286745c08a31fe5a86b2dc26" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 35491 } } peers { permanent_uuid: "615975d774ee48f1a56e2286986768b7" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 44537 } }
I20260924 21:16:06.356755 24456 ts_tablet_manager.cc:1428] T cfb7748b170f4ee1a8b91f2a40b80db3 P 623a14d221f1410aa430e37f39a36e56: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260924 21:16:06.356777 24438 heartbeater.cc:503] Master 127.6.210.190:37823 was elected leader, sending a full tablet report...
I20260924 21:16:06.368450 24462 raft_consensus.cc:482] T cfb7748b170f4ee1a8b91f2a40b80db3 P 40a0ebe1286745c08a31fe5a86b2dc26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:16:06.368559 24462 raft_consensus.cc:504] T cfb7748b170f4ee1a8b91f2a40b80db3 P 40a0ebe1286745c08a31fe5a86b2dc26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623a14d221f1410aa430e37f39a36e56" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 33639 } } peers { permanent_uuid: "40a0ebe1286745c08a31fe5a86b2dc26" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 35491 } } peers { permanent_uuid: "615975d774ee48f1a56e2286986768b7" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 44537 } }
I20260924 21:16:06.368880 24462 leader_election.cc:288] T cfb7748b170f4ee1a8b91f2a40b80db3 P 40a0ebe1286745c08a31fe5a86b2dc26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 623a14d221f1410aa430e37f39a36e56 (127.6.210.131:33639), 615975d774ee48f1a56e2286986768b7 (127.6.210.129:44537)
I20260924 21:16:06.371737 24306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:06.369075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:44537 (local address 127.6.210.130:50811)
0924 21:16:06.369132 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:06.369143 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:06.369209 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:06.369356 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:06.369357 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:06.369359 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:06.369444 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:06.369445 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.370070 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.370071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:06.370860 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:06.370862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.371439 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.371440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:06.371521 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:06.371529 (+ 8us) client_negotiation.cc:769] Sending connection context
0924 21:16:06.371615 (+ 86us) client_negotiation.cc:240] Negotiation successful
0924 21:16:06.371658 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260924 21:16:06.371784 24453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:06.369143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:50811 (local address 127.6.210.129:44537)
0924 21:16:06.369183 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:06.369185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:06.369193 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:16:06.369271 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:06.369272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:06.369281 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:06.369354 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:06.369516 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.369982 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.370966 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.371346 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.371441 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:06.371450 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:06.371636 (+ 186us) server_negotiation.cc:299] Negotiation successful
0924 21:16:06.371689 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 21:16:06.371935 24454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:06.369181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:60363 (local address 127.6.210.131:33639)
0924 21:16:06.369228 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:06.369230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:06.369380 (+ 150us) server_negotiation.cc:407] Connection header received
0924 21:16:06.369484 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:06.369485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:06.369495 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:06.369570 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:06.369731 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.370187 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.371110 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.371482 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.371579 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:06.371586 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:06.371816 (+ 230us) server_negotiation.cc:299] Negotiation successful
0924 21:16:06.371863 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260924 21:16:06.372151 24464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:06.369118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:33639 (local address 127.6.210.130:60363)
0924 21:16:06.369301 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:06.369310 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:06.369394 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:06.369577 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:06.369578 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:06.369579 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:06.369652 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:06.369653 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.370285 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.370286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:06.371006 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:06.371008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.371581 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.371582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:06.371729 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:06.371733 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:16:06.372040 (+ 307us) client_negotiation.cc:240] Negotiation successful
0924 21:16:06.372080 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":122,"threads_started":1}
I20260924 21:16:06.372136 24124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb7748b170f4ee1a8b91f2a40b80db3" candidate_uuid: "40a0ebe1286745c08a31fe5a86b2dc26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615975d774ee48f1a56e2286986768b7" is_pre_election: true
I20260924 21:16:06.372310 24124 raft_consensus.cc:2454] T cfb7748b170f4ee1a8b91f2a40b80db3 P 615975d774ee48f1a56e2286986768b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40a0ebe1286745c08a31fe5a86b2dc26 in term 0.
I20260924 21:16:06.372498 24392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb7748b170f4ee1a8b91f2a40b80db3" candidate_uuid: "40a0ebe1286745c08a31fe5a86b2dc26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "623a14d221f1410aa430e37f39a36e56" is_pre_election: true
I20260924 21:16:06.372575 24192 leader_election.cc:302] T cfb7748b170f4ee1a8b91f2a40b80db3 P 40a0ebe1286745c08a31fe5a86b2dc26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40a0ebe1286745c08a31fe5a86b2dc26, 615975d774ee48f1a56e2286986768b7; no voters:
I20260924 21:16:06.372655 24392 raft_consensus.cc:2454] T cfb7748b170f4ee1a8b91f2a40b80db3 P 623a14d221f1410aa430e37f39a36e56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40a0ebe1286745c08a31fe5a86b2dc26 in term 0.
I20260924 21:16:06.372789 24462 raft_consensus.cc:2781] T cfb7748b170f4ee1a8b91f2a40b80db3 P 40a0ebe1286745c08a31fe5a86b2dc26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:16:06.372853 24462 raft_consensus.cc:482] T cfb7748b170f4ee1a8b91f2a40b80db3 P 40a0ebe1286745c08a31fe5a86b2dc26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:16:06.372876 24462 raft_consensus.cc:3037] T cfb7748b170f4ee1a8b91f2a40b80db3 P 40a0ebe1286745c08a31fe5a86b2dc26 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:06.373663 24462 raft_consensus.cc:504] T cfb7748b170f4ee1a8b91f2a40b80db3 P 40a0ebe1286745c08a31fe5a86b2dc26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623a14d221f1410aa430e37f39a36e56" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 33639 } } peers { permanent_uuid: "40a0ebe1286745c08a31fe5a86b2dc26" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 35491 } } peers { permanent_uuid: "615975d774ee48f1a56e2286986768b7" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 44537 } }
I20260924 21:16:06.373804 24462 leader_election.cc:288] T cfb7748b170f4ee1a8b91f2a40b80db3 P 40a0ebe1286745c08a31fe5a86b2dc26 [CANDIDATE]: Term 1 election: Requested vote from peers 623a14d221f1410aa430e37f39a36e56 (127.6.210.131:33639), 615975d774ee48f1a56e2286986768b7 (127.6.210.129:44537)
I20260924 21:16:06.373953 24124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb7748b170f4ee1a8b91f2a40b80db3" candidate_uuid: "40a0ebe1286745c08a31fe5a86b2dc26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "615975d774ee48f1a56e2286986768b7"
I20260924 21:16:06.373971 24392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfb7748b170f4ee1a8b91f2a40b80db3" candidate_uuid: "40a0ebe1286745c08a31fe5a86b2dc26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "623a14d221f1410aa430e37f39a36e56"
I20260924 21:16:06.374044 24124 raft_consensus.cc:3037] T cfb7748b170f4ee1a8b91f2a40b80db3 P 615975d774ee48f1a56e2286986768b7 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:06.374040 24392 raft_consensus.cc:3037] T cfb7748b170f4ee1a8b91f2a40b80db3 P 623a14d221f1410aa430e37f39a36e56 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:06.374828 24124 raft_consensus.cc:2454] T cfb7748b170f4ee1a8b91f2a40b80db3 P 615975d774ee48f1a56e2286986768b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40a0ebe1286745c08a31fe5a86b2dc26 in term 1.
I20260924 21:16:06.374907 24392 raft_consensus.cc:2454] T cfb7748b170f4ee1a8b91f2a40b80db3 P 623a14d221f1410aa430e37f39a36e56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40a0ebe1286745c08a31fe5a86b2dc26 in term 1.
I20260924 21:16:06.375041 24192 leader_election.cc:302] T cfb7748b170f4ee1a8b91f2a40b80db3 P 40a0ebe1286745c08a31fe5a86b2dc26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40a0ebe1286745c08a31fe5a86b2dc26, 615975d774ee48f1a56e2286986768b7; no voters:
I20260924 21:16:06.375121 24462 raft_consensus.cc:2781] T cfb7748b170f4ee1a8b91f2a40b80db3 P 40a0ebe1286745c08a31fe5a86b2dc26 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:06.375272 24462 raft_consensus.cc:686] T cfb7748b170f4ee1a8b91f2a40b80db3 P 40a0ebe1286745c08a31fe5a86b2dc26 [term 1 LEADER]: Becoming Leader. State: Replica: 40a0ebe1286745c08a31fe5a86b2dc26, State: Running, Role: LEADER
I20260924 21:16:06.375384 24462 consensus_queue.cc:237] T cfb7748b170f4ee1a8b91f2a40b80db3 P 40a0ebe1286745c08a31fe5a86b2dc26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623a14d221f1410aa430e37f39a36e56" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 33639 } } peers { permanent_uuid: "40a0ebe1286745c08a31fe5a86b2dc26" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 35491 } } peers { permanent_uuid: "615975d774ee48f1a56e2286986768b7" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 44537 } }
I20260924 21:16:06.376230 23978 catalog_manager.cc:5495] T cfb7748b170f4ee1a8b91f2a40b80db3 P 40a0ebe1286745c08a31fe5a86b2dc26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 40a0ebe1286745c08a31fe5a86b2dc26 (127.6.210.130). New cstate: current_term: 1 leader_uuid: "40a0ebe1286745c08a31fe5a86b2dc26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623a14d221f1410aa430e37f39a36e56" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 33639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40a0ebe1286745c08a31fe5a86b2dc26" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 35491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "615975d774ee48f1a56e2286986768b7" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 44537 } health_report { overall_health: UNKNOWN } } }
I20260924 21:16:06.383991 24452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:06.381632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47652 (local address 127.6.210.130:35491)
0924 21:16:06.381686 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:06.381688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:06.381746 (+ 58us) server_negotiation.cc:407] Connection header received
0924 21:16:06.381813 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:06.381814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:06.381827 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:06.381885 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:16:06.382056 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.382557 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.383292 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.383468 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.383536 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:06.383695 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:16:06.383784 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:16:06.383842 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:06.383899 (+ 57us) server_negotiation.cc:299] Negotiation successful
0924 21:16:06.383916 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 21:16:06.385775 24124 raft_consensus.cc:1261] T cfb7748b170f4ee1a8b91f2a40b80db3 P 615975d774ee48f1a56e2286986768b7 [term 1 FOLLOWER]: Refusing update from remote peer 40a0ebe1286745c08a31fe5a86b2dc26: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:16:06.385821 24392 raft_consensus.cc:1261] T cfb7748b170f4ee1a8b91f2a40b80db3 P 623a14d221f1410aa430e37f39a36e56 [term 1 FOLLOWER]: Refusing update from remote peer 40a0ebe1286745c08a31fe5a86b2dc26: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:16:06.386142 24465 consensus_queue.cc:1035] T cfb7748b170f4ee1a8b91f2a40b80db3 P 40a0ebe1286745c08a31fe5a86b2dc26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "615975d774ee48f1a56e2286986768b7" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 44537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:16:06.386250 24462 consensus_queue.cc:1035] T cfb7748b170f4ee1a8b91f2a40b80db3 P 40a0ebe1286745c08a31fe5a86b2dc26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "623a14d221f1410aa430e37f39a36e56" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 33639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:16:06.388710 24468 mvcc.cc:205] Tried to move back new op lower bound from 7333005583913451520 to 7333005583874220032. Current Snapshot: MvccSnapshot[applied={T|T < 7333005583913451520}]
I20260924 21:16:06.389055 24469 mvcc.cc:205] Tried to move back new op lower bound from 7333005583913451520 to 7333005583874220032. Current Snapshot: MvccSnapshot[applied={T|T < 7333005583913451520}]
I20260924 21:16:06.389339 24470 mvcc.cc:205] Tried to move back new op lower bound from 7333005583913451520 to 7333005583874220032. Current Snapshot: MvccSnapshot[applied={T|T < 7333005583913451520 or (T in {7333005583913451520})}]
I20260924 21:16:06.391445 23979 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:16:06.391569 23979 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:16:06.392287 23979 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a774b6f451174827aec721486c2be0f1: Sending DeleteTablet for 3 replicas of tablet cfb7748b170f4ee1a8b91f2a40b80db3
I20260924 21:16:06.392611 24372 tablet_service.cc:1514] Processing DeleteTablet for tablet cfb7748b170f4ee1a8b91f2a40b80db3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:06 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:41840
I20260924 21:16:06.392642 24104 tablet_service.cc:1514] Processing DeleteTablet for tablet cfb7748b170f4ee1a8b91f2a40b80db3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:06 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:58046
I20260924 21:16:06.392699 24238 tablet_service.cc:1514] Processing DeleteTablet for tablet cfb7748b170f4ee1a8b91f2a40b80db3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:06 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:47638
I20260924 21:16:06.392879 24478 tablet_replica.cc:330] T cfb7748b170f4ee1a8b91f2a40b80db3 P 615975d774ee48f1a56e2286986768b7: stopping tablet replica
I20260924 21:16:06.392882 24477 tablet_replica.cc:330] T cfb7748b170f4ee1a8b91f2a40b80db3 P 623a14d221f1410aa430e37f39a36e56: stopping tablet replica
I20260924 21:16:06.392977 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 24041
I20260924 21:16:06.393038 24478 raft_consensus.cc:2229] T cfb7748b170f4ee1a8b91f2a40b80db3 P 615975d774ee48f1a56e2286986768b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:16:06.393095 24477 raft_consensus.cc:2229] T cfb7748b170f4ee1a8b91f2a40b80db3 P 623a14d221f1410aa430e37f39a36e56 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:16:06.393664 24477 raft_consensus.cc:2258] T cfb7748b170f4ee1a8b91f2a40b80db3 P 623a14d221f1410aa430e37f39a36e56 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:16:06.393803 24479 tablet_replica.cc:330] T cfb7748b170f4ee1a8b91f2a40b80db3 P 40a0ebe1286745c08a31fe5a86b2dc26: stopping tablet replica
I20260924 21:16:06.393882 24479 raft_consensus.cc:2229] T cfb7748b170f4ee1a8b91f2a40b80db3 P 40a0ebe1286745c08a31fe5a86b2dc26 [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:16:06.393990 24479 raft_consensus.cc:2258] T cfb7748b170f4ee1a8b91f2a40b80db3 P 40a0ebe1286745c08a31fe5a86b2dc26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:16:06.394374 24479 ts_tablet_manager.cc:1905] T cfb7748b170f4ee1a8b91f2a40b80db3 P 40a0ebe1286745c08a31fe5a86b2dc26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:16:06.394831 24477 ts_tablet_manager.cc:1905] T cfb7748b170f4ee1a8b91f2a40b80db3 P 623a14d221f1410aa430e37f39a36e56: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:16:06.396281 24479 ts_tablet_manager.cc:1918] T cfb7748b170f4ee1a8b91f2a40b80db3 P 40a0ebe1286745c08a31fe5a86b2dc26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:16:06.396395 24479 log.cc:1196] T cfb7748b170f4ee1a8b91f2a40b80db3 P 40a0ebe1286745c08a31fe5a86b2dc26: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-1/wal/wals/cfb7748b170f4ee1a8b91f2a40b80db3
I20260924 21:16:06.396545 24477 ts_tablet_manager.cc:1918] T cfb7748b170f4ee1a8b91f2a40b80db3 P 623a14d221f1410aa430e37f39a36e56: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:16:06.396613 24477 log.cc:1196] T cfb7748b170f4ee1a8b91f2a40b80db3 P 623a14d221f1410aa430e37f39a36e56: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284333163253-6986-0/minicluster-data/ts-2/wal/wals/cfb7748b170f4ee1a8b91f2a40b80db3
I20260924 21:16:06.396886 24477 ts_tablet_manager.cc:1939] T cfb7748b170f4ee1a8b91f2a40b80db3 P 623a14d221f1410aa430e37f39a36e56: Deleting consensus metadata
I20260924 21:16:06.397332 23966 catalog_manager.cc:4841] TS 623a14d221f1410aa430e37f39a36e56 (127.6.210.131:33639): tablet cfb7748b170f4ee1a8b91f2a40b80db3 (table test-table [id=7fe5712b8ef34e959ae486b0c2ba2b6f]) successfully deleted
I20260924 21:16:06.397449 24479 ts_tablet_manager.cc:1939] T cfb7748b170f4ee1a8b91f2a40b80db3 P 40a0ebe1286745c08a31fe5a86b2dc26: Deleting consensus metadata
I20260924 21:16:06.397859 23965 catalog_manager.cc:4841] TS 40a0ebe1286745c08a31fe5a86b2dc26 (127.6.210.130:35491): tablet cfb7748b170f4ee1a8b91f2a40b80db3 (table test-table [id=7fe5712b8ef34e959ae486b0c2ba2b6f]) successfully deleted
W20260924 21:16:06.399401 23964 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.210.129:44537 (error 108)
I20260924 21:16:06.399652 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 24175
I20260924 21:16:06.402082 24450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:06.401947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:44537 (local address 127.0.0.1:58056)
0924 21:16:06.402008 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:06.402054 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.129:44537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260924 21:16:06.402210 23964 catalog_manager.cc:4584] TS 615975d774ee48f1a56e2286986768b7 (127.6.210.129:44537): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cfb7748b170f4ee1a8b91f2a40b80db3: Network error: Client connection negotiation failed: client connection to 127.6.210.129:44537: connect: Connection refused (error 111)
I20260924 21:16:06.405308 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 24309
I20260924 21:16:06.409699 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 23947
2026-09-24T21:16:06Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1962 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-24T21:16:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:16:06Z Disabled control of system clock
I20260924 21:16:06.437121 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:36435
--webserver_interface=127.6.210.190
--webserver_port=0
--builtin_ntp_servers=127.6.210.148:36019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:36435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 21:16:06.518602 24484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:06.518846 24484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:06.518867 24484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:06.520145 24484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:06.520191 24484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:06.520203 24484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:06.520216 24484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:06.520228 24484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:06.521687 24484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:36019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:36435
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:36435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:06.521948 24484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:06.522244 24484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:06.527060 24490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:06.527065 24492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:06.527067 24489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:06.527480 24484 server_base.cc:1034] running on GCE node
I20260924 21:16:06.527693 24484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:06.527932 24484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:06.529093 24484 hybrid_clock.cc:648] HybridClock initialized: now 1790284566529061 us; error 56 us; skew 500 ppm
I20260924 21:16:06.530242 24484 webserver.cc:460] Webserver started at http://127.6.210.190:34649/ using document root <none> and password file <none>
I20260924 21:16:06.530536 24484 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:06.530587 24484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:06.530678 24484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:06.531740 24484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "d521507173a7425a92d559a59624a42c"
format_stamp: "Formatted at 2026-09-24 21:16:06 on dist-test-slave-q276"
server_key: "d7fb5c4a108a01ed7022fa4fad526692"
server_key_iv: "4ebfaa932c92c833a166750364ed9938"
server_key_version: "encryptionkey@0"
I20260924 21:16:06.532044 24484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "d521507173a7425a92d559a59624a42c"
format_stamp: "Formatted at 2026-09-24 21:16:06 on dist-test-slave-q276"
server_key: "d7fb5c4a108a01ed7022fa4fad526692"
server_key_iv: "4ebfaa932c92c833a166750364ed9938"
server_key_version: "encryptionkey@0"
I20260924 21:16:06.533218 24484 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:16:06.534001 24498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:06.534220 24484 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:06.534350 24484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "d521507173a7425a92d559a59624a42c"
format_stamp: "Formatted at 2026-09-24 21:16:06 on dist-test-slave-q276"
server_key: "d7fb5c4a108a01ed7022fa4fad526692"
server_key_iv: "4ebfaa932c92c833a166750364ed9938"
server_key_version: "encryptionkey@0"
I20260924 21:16:06.534411 24484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:06.551697 24484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:06.551976 24484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:06.552091 24484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:06.556174 24484 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:36435
I20260924 21:16:06.556241 24550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:36435 every 8 connection(s)
I20260924 21:16:06.556594 24484 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:16:06.557593 24551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:06.559872 24551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d521507173a7425a92d559a59624a42c: Bootstrap starting.
I20260924 21:16:06.560396 24551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d521507173a7425a92d559a59624a42c: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:06.560642 24551 log.cc:824] T 00000000000000000000000000000000 P d521507173a7425a92d559a59624a42c: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:06.561281 24551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d521507173a7425a92d559a59624a42c: No bootstrap required, opened a new log
I20260924 21:16:06.561611 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 24484
I20260924 21:16:06.561715 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:16:06.561897 6986 external_mini_cluster.cc:1428] Setting key fdd176603aa02bc75a08d06587784cb8
I20260924 21:16:06.563346 24551 raft_consensus.cc:348] T 00000000000000000000000000000000 P d521507173a7425a92d559a59624a42c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d521507173a7425a92d559a59624a42c" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36435 } }
I20260924 21:16:06.563534 24551 raft_consensus.cc:374] T 00000000000000000000000000000000 P d521507173a7425a92d559a59624a42c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:06.563650 24551 raft_consensus.cc:729] T 00000000000000000000000000000000 P d521507173a7425a92d559a59624a42c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d521507173a7425a92d559a59624a42c, State: Initialized, Role: FOLLOWER
I20260924 21:16:06.563799 24551 consensus_queue.cc:260] T 00000000000000000000000000000000 P d521507173a7425a92d559a59624a42c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d521507173a7425a92d559a59624a42c" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36435 } }
I20260924 21:16:06.563910 24551 raft_consensus.cc:388] T 00000000000000000000000000000000 P d521507173a7425a92d559a59624a42c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:16:06.563952 24551 raft_consensus.cc:482] T 00000000000000000000000000000000 P d521507173a7425a92d559a59624a42c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:16:06.564002 24551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d521507173a7425a92d559a59624a42c [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:06.564545 24551 raft_consensus.cc:504] T 00000000000000000000000000000000 P d521507173a7425a92d559a59624a42c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d521507173a7425a92d559a59624a42c" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36435 } }
I20260924 21:16:06.564667 24551 leader_election.cc:302] T 00000000000000000000000000000000 P d521507173a7425a92d559a59624a42c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d521507173a7425a92d559a59624a42c; no voters:
I20260924 21:16:06.564853 24551 leader_election.cc:288] T 00000000000000000000000000000000 P d521507173a7425a92d559a59624a42c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:16:06.564883 24556 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d521507173a7425a92d559a59624a42c [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:06.565016 24556 raft_consensus.cc:686] T 00000000000000000000000000000000 P d521507173a7425a92d559a59624a42c [term 1 LEADER]: Becoming Leader. State: Replica: d521507173a7425a92d559a59624a42c, State: Running, Role: LEADER
I20260924 21:16:06.565105 24551 sys_catalog.cc:564] T 00000000000000000000000000000000 P d521507173a7425a92d559a59624a42c [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:06.565145 24556 consensus_queue.cc:237] T 00000000000000000000000000000000 P d521507173a7425a92d559a59624a42c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d521507173a7425a92d559a59624a42c" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36435 } }
I20260924 21:16:06.565538 24556 sys_catalog.cc:455] T 00000000000000000000000000000000 P d521507173a7425a92d559a59624a42c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d521507173a7425a92d559a59624a42c. Latest consensus state: current_term: 1 leader_uuid: "d521507173a7425a92d559a59624a42c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d521507173a7425a92d559a59624a42c" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36435 } } }
I20260924 21:16:06.565680 24556 sys_catalog.cc:458] T 00000000000000000000000000000000 P d521507173a7425a92d559a59624a42c [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:06.566141 24565 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:16:06.566491 24558 sys_catalog.cc:455] T 00000000000000000000000000000000 P d521507173a7425a92d559a59624a42c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d521507173a7425a92d559a59624a42c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d521507173a7425a92d559a59624a42c" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36435 } } }
I20260924 21:16:06.566589 24558 sys_catalog.cc:458] T 00000000000000000000000000000000 P d521507173a7425a92d559a59624a42c [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:06.566746 24565 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:16:06.568140 24565 catalog_manager.cc:1347] Generated new cluster ID: 6a1ba2f006d84bc2b1d4847f89717d29
I20260924 21:16:06.568190 24565 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:16:06.569021 24555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:06.562427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50402 (local address 127.6.210.190:36435)
0924 21:16:06.562624 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:06.562632 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:06.562656 (+ 24us) server_negotiation.cc:407] Connection header received
0924 21:16:06.563325 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:06.563332 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:06.563389 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:06.563501 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:06.563802 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.565320 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.566583 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.567181 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.567272 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:06.568297 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:06.568301 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:06.568302 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:06.568495 (+ 193us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:06.568540 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:06.568555 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:06.568618 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:06.568753 (+ 135us) server_negotiation.cc:299] Negotiation successful
0924 21:16:06.568827 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":107,"threads_started":1}
I20260924 21:16:06.593186 24565 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:16:06.593927 24565 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:16:06.607841 24565 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d521507173a7425a92d559a59624a42c: Generated new TSK 0
I20260924 21:16:06.608100 24565 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:16:06.612210 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.129:0
--local_ip_for_outbound_sockets=127.6.210.129
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:36435
--builtin_ntp_servers=127.6.210.148:36019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 21:16:06.700124 24575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:06.700325 24575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:06.700350 24575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:06.701463 24575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:06.701498 24575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:06.701522 24575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.129
I20260924 21:16:06.702879 24575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:36019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:36435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.129
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:06.703082 24575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:06.703329 24575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 21:16:06.708832 24575 server_base.cc:1034] running on GCE node
W20260924 21:16:06.708784 24580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:06.708752 24583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:06.708755 24581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:06.709393 24575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:06.709626 24575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:06.710769 24575 hybrid_clock.cc:648] HybridClock initialized: now 1790284566710758 us; error 31 us; skew 500 ppm
I20260924 21:16:06.711998 24575 webserver.cc:460] Webserver started at http://127.6.210.129:43395/ using document root <none> and password file <none>
I20260924 21:16:06.712249 24575 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:06.712296 24575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:06.712482 24575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:06.713552 24575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-0/data/instance:
uuid: "085bbbf35613436ea3e80a7624832eb2"
format_stamp: "Formatted at 2026-09-24 21:16:06 on dist-test-slave-q276"
server_key: "107d9ad5b7170e3d90fc0bcdff5117fd"
server_key_iv: "7bb3f3bb2920cc8a35d4308cc0427dff"
server_key_version: "encryptionkey@0"
I20260924 21:16:06.713948 24575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance:
uuid: "085bbbf35613436ea3e80a7624832eb2"
format_stamp: "Formatted at 2026-09-24 21:16:06 on dist-test-slave-q276"
server_key: "107d9ad5b7170e3d90fc0bcdff5117fd"
server_key_iv: "7bb3f3bb2920cc8a35d4308cc0427dff"
server_key_version: "encryptionkey@0"
I20260924 21:16:06.715345 24575 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:16:06.716151 24589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:06.716454 24575 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:16:06.716557 24575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-0/wal
uuid: "085bbbf35613436ea3e80a7624832eb2"
format_stamp: "Formatted at 2026-09-24 21:16:06 on dist-test-slave-q276"
server_key: "107d9ad5b7170e3d90fc0bcdff5117fd"
server_key_iv: "7bb3f3bb2920cc8a35d4308cc0427dff"
server_key_version: "encryptionkey@0"
I20260924 21:16:06.716637 24575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:06.736450 24575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:06.736768 24575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:06.736943 24575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:06.737188 24575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:16:06.737504 24575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:16:06.737560 24575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:06.737601 24575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:16:06.737627 24575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:06.743366 24575 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.129:39471
I20260924 21:16:06.743422 24702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.129:39471 every 8 connection(s)
I20260924 21:16:06.743774 24575 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
I20260924 21:16:06.747192 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 24575
I20260924 21:16:06.747270 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance
I20260924 21:16:06.747453 6986 external_mini_cluster.cc:1428] Setting key 3a57b0ff9d3d2417bad621e7d57b3dd7
I20260924 21:16:06.748461 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.130:0
--local_ip_for_outbound_sockets=127.6.210.130
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:36435
--builtin_ntp_servers=127.6.210.148:36019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260924 21:16:06.749255 24555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:06.744605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:53583 (local address 127.6.210.190:36435)
0924 21:16:06.744675 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:06.744677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:06.744922 (+ 245us) server_negotiation.cc:407] Connection header received
0924 21:16:06.745385 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:06.745386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:06.745395 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:06.745457 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:06.746091 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.746633 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.747738 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.748003 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.748082 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:06.748969 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:06.748971 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:06.748971 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:06.748976 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:06.748992 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:06.748997 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:06.749061 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:06.749111 (+ 50us) server_negotiation.cc:299] Negotiation successful
0924 21:16:06.749139 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 21:16:06.749379 24705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:06.744605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36435 (local address 127.6.210.129:53583)
0924 21:16:06.744794 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:06.744834 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:06.745302 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:06.745511 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:06.745514 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:06.745522 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:06.746017 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:06.746021 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.746708 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.746710 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:06.747648 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:06.747650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.748092 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.748093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:06.748219 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:06.748833 (+ 614us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:16:06.748833 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:06.748862 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:16:06.748883 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:06.749062 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:06.749064 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:06.749066 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 21:16:06.749112 (+ 46us) client_negotiation.cc:240] Negotiation successful
0924 21:16:06.749185 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
I20260924 21:16:06.749866 24703 heartbeater.cc:347] Connected to a master server at 127.6.210.190:36435
I20260924 21:16:06.749974 24703 heartbeater.cc:464] Registering TS with master...
I20260924 21:16:06.750144 24703 heartbeater.cc:511] Master 127.6.210.190:36435 requested a full tablet report, sending...
I20260924 21:16:06.750799 24515 ts_manager.cc:194] Registered new tserver with Master: 085bbbf35613436ea3e80a7624832eb2 (127.6.210.129:39471)
I20260924 21:16:06.751405 24515 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.210.129:53583
W20260924 21:16:06.830751 24706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:06.830955 24706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:06.830977 24706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:06.832067 24706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:06.832101 24706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:06.832125 24706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.130
I20260924 21:16:06.833375 24706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:36019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:36435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.130
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:06.833570 24706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:06.833806 24706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:06.838884 24712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:06.838860 24711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:06.838860 24714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:06.838996 24706 server_base.cc:1034] running on GCE node
I20260924 21:16:06.839283 24706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:06.839486 24706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:06.840624 24706 hybrid_clock.cc:648] HybridClock initialized: now 1790284566840611 us; error 30 us; skew 500 ppm
I20260924 21:16:06.841830 24706 webserver.cc:460] Webserver started at http://127.6.210.130:42199/ using document root <none> and password file <none>
I20260924 21:16:06.842116 24706 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:06.842195 24706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:06.842360 24706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:06.843398 24706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-1/data/instance:
uuid: "5da66b3225354b5b94b0a340dd17a362"
format_stamp: "Formatted at 2026-09-24 21:16:06 on dist-test-slave-q276"
server_key: "46de301c5d40673be4183673d0809825"
server_key_iv: "6d2064543eb4f6e97610e13e87f1b1f4"
server_key_version: "encryptionkey@0"
I20260924 21:16:06.843756 24706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance:
uuid: "5da66b3225354b5b94b0a340dd17a362"
format_stamp: "Formatted at 2026-09-24 21:16:06 on dist-test-slave-q276"
server_key: "46de301c5d40673be4183673d0809825"
server_key_iv: "6d2064543eb4f6e97610e13e87f1b1f4"
server_key_version: "encryptionkey@0"
I20260924 21:16:06.845104 24706 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:16:06.845870 24720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:06.846144 24706 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:06.846246 24706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-1/wal
uuid: "5da66b3225354b5b94b0a340dd17a362"
format_stamp: "Formatted at 2026-09-24 21:16:06 on dist-test-slave-q276"
server_key: "46de301c5d40673be4183673d0809825"
server_key_iv: "6d2064543eb4f6e97610e13e87f1b1f4"
server_key_version: "encryptionkey@0"
I20260924 21:16:06.846376 24706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:06.864262 24706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:06.864600 24706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:06.864784 24706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:06.865031 24706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:16:06.865391 24706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:16:06.865448 24706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:06.865490 24706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:16:06.865518 24706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:06.871625 24706 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.130:38563
I20260924 21:16:06.871670 24833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.130:38563 every 8 connection(s)
I20260924 21:16:06.872058 24706 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
I20260924 21:16:06.874780 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 24706
I20260924 21:16:06.874877 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance
I20260924 21:16:06.875074 6986 external_mini_cluster.cc:1428] Setting key 6cf41a36776a4d11ce321c59faaab20f
I20260924 21:16:06.876052 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.131:0
--local_ip_for_outbound_sockets=127.6.210.131
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:36435
--builtin_ntp_servers=127.6.210.148:36019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260924 21:16:06.878389 24555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:06.872837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:56521 (local address 127.6.210.190:36435)
0924 21:16:06.872908 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:06.872910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:06.873185 (+ 275us) server_negotiation.cc:407] Connection header received
0924 21:16:06.873686 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:06.873687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:06.873697 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:06.873751 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:06.874522 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.875289 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.876835 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.877025 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.877089 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:06.878063 (+ 974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:06.878066 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:06.878066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:06.878072 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:06.878090 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:06.878095 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:06.878158 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:06.878254 (+ 96us) server_negotiation.cc:299] Negotiation successful
0924 21:16:06.878317 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 21:16:06.878647 24836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:06.872836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36435 (local address 127.6.210.130:56521)
0924 21:16:06.873030 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:06.873074 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:06.873590 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:06.873823 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:06.873827 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:06.873835 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:06.874411 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:06.874416 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.875393 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.875394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:06.876657 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:06.876660 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.877115 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.877117 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:06.877285 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:06.877896 (+ 611us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:16:06.877897 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:06.877926 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:16:06.877950 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:06.878167 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:06.878171 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:06.878174 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:16:06.878230 (+ 56us) client_negotiation.cc:240] Negotiation successful
0924 21:16:06.878379 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":70,"threads_started":1}
I20260924 21:16:06.879217 24834 heartbeater.cc:347] Connected to a master server at 127.6.210.190:36435
I20260924 21:16:06.879321 24834 heartbeater.cc:464] Registering TS with master...
I20260924 21:16:06.879523 24834 heartbeater.cc:511] Master 127.6.210.190:36435 requested a full tablet report, sending...
I20260924 21:16:06.879882 24515 ts_manager.cc:194] Registered new tserver with Master: 5da66b3225354b5b94b0a340dd17a362 (127.6.210.130:38563)
I20260924 21:16:06.880264 24515 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.210.130:56521
W20260924 21:16:06.957504 24837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:06.957723 24837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:06.957746 24837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:06.959004 24837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:06.959053 24837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:06.959079 24837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.131
I20260924 21:16:06.960448 24837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:36019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:36435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.131
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:06.960700 24837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:06.960995 24837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:06.965983 24842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:06.965992 24843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:06.966120 24837 server_base.cc:1034] running on GCE node
W20260924 21:16:06.965984 24845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:06.966475 24837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:06.966728 24837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:06.967885 24837 hybrid_clock.cc:648] HybridClock initialized: now 1790284566967880 us; error 35 us; skew 500 ppm
I20260924 21:16:06.969111 24837 webserver.cc:460] Webserver started at http://127.6.210.131:35739/ using document root <none> and password file <none>
I20260924 21:16:06.969352 24837 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:06.969424 24837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:06.969560 24837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:06.970697 24837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-2/data/instance:
uuid: "fba7d076c379415f8264da94fd1f7601"
format_stamp: "Formatted at 2026-09-24 21:16:06 on dist-test-slave-q276"
server_key: "25e1b11344059fb5ebcb3baf81efb1e5"
server_key_iv: "9c28974937b9ee0e3b8a57672bebb774"
server_key_version: "encryptionkey@0"
I20260924 21:16:06.971060 24837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance:
uuid: "fba7d076c379415f8264da94fd1f7601"
format_stamp: "Formatted at 2026-09-24 21:16:06 on dist-test-slave-q276"
server_key: "25e1b11344059fb5ebcb3baf81efb1e5"
server_key_iv: "9c28974937b9ee0e3b8a57672bebb774"
server_key_version: "encryptionkey@0"
I20260924 21:16:06.972337 24837 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 21:16:06.973080 24851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:06.973263 24837 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:16:06.973364 24837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-2/wal
uuid: "fba7d076c379415f8264da94fd1f7601"
format_stamp: "Formatted at 2026-09-24 21:16:06 on dist-test-slave-q276"
server_key: "25e1b11344059fb5ebcb3baf81efb1e5"
server_key_iv: "9c28974937b9ee0e3b8a57672bebb774"
server_key_version: "encryptionkey@0"
I20260924 21:16:06.973446 24837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:06.982474 24837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:06.982749 24837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:06.982896 24837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:06.983124 24837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:16:06.983438 24837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:16:06.983493 24837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:06.983532 24837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:16:06.983559 24837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:06.989048 24837 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.131:45751
I20260924 21:16:06.989114 24964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.131:45751 every 8 connection(s)
I20260924 21:16:06.989452 24837 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
I20260924 21:16:06.991626 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 24837
I20260924 21:16:06.991732 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance
I20260924 21:16:06.991971 6986 external_mini_cluster.cc:1428] Setting key 0fcb9b396e2fb59fc1e11185abc59bcf
I20260924 21:16:06.994966 24555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:06.990257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:48825 (local address 127.6.210.190:36435)
0924 21:16:06.990348 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:06.990350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:06.990623 (+ 273us) server_negotiation.cc:407] Connection header received
0924 21:16:06.991079 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:06.991080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:06.991089 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:06.991154 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:06.991877 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.992659 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.993640 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.993819 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.993871 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:06.994755 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:06.994756 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:06.994757 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:06.994760 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:06.994773 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:06.994777 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:06.994818 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:06.994888 (+ 70us) server_negotiation.cc:299] Negotiation successful
0924 21:16:06.994909 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 21:16:06.995203 24967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:06.990251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36435 (local address 127.6.210.131:48825)
0924 21:16:06.990464 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:06.990505 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:06.990989 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:06.991210 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:06.991214 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:06.991221 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:06.991714 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:06.991718 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.992751 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.992752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:06.993565 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:06.993567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:06.993889 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:06.993890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:06.993992 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:06.994623 (+ 631us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:16:06.994623 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:06.994651 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:16:06.994673 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:06.994825 (+ 152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:06.994827 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:06.994829 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 21:16:06.994875 (+ 46us) client_negotiation.cc:240] Negotiation successful
0924 21:16:06.994947 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":98,"threads_started":1}
I20260924 21:16:06.995729 24965 heartbeater.cc:347] Connected to a master server at 127.6.210.190:36435
I20260924 21:16:06.995828 24965 heartbeater.cc:464] Registering TS with master...
I20260924 21:16:06.996001 24965 heartbeater.cc:511] Master 127.6.210.190:36435 requested a full tablet report, sending...
I20260924 21:16:06.996358 24515 ts_manager.cc:194] Registered new tserver with Master: fba7d076c379415f8264da94fd1f7601 (127.6.210.131:45751)
I20260924 21:16:06.996680 24515 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.210.131:48825
I20260924 21:16:07.003660 6986 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:16:07.008081 24555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:07.005134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50406 (local address 127.6.210.190:36435)
0924 21:16:07.005204 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:07.005206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:07.005308 (+ 102us) server_negotiation.cc:407] Connection header received
0924 21:16:07.005354 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:07.005355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:07.005365 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:07.005421 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:07.005577 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:07.006035 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:07.006756 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:07.006949 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:07.007001 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:07.007746 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:07.007747 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:07.007747 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:07.007758 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:07.007774 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:07.007779 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:07.007837 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:07.007947 (+ 110us) server_negotiation.cc:299] Negotiation successful
0924 21:16:07.007977 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 21:16:07.009907 24515 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50406:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:16:07.011530 24515 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:16:07.017959 24977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:07.013812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:39471 (local address 127.0.0.1:51524)
0924 21:16:07.014121 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:07.014131 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:07.014223 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:07.014582 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:07.014585 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:07.014588 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:07.014670 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:07.014672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:07.015596 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:07.015597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:07.016670 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:07.016672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:07.017652 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:07.017653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:07.017735 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:07.017749 (+ 14us) client_negotiation.cc:769] Sending connection context
0924 21:16:07.017831 (+ 82us) client_negotiation.cc:240] Negotiation successful
0924 21:16:07.017878 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":4,"thread_start_us":99,"threads_started":1}
I20260924 21:16:07.018303 24974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:07.013802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:38563 (local address 127.0.0.1:59274)
0924 21:16:07.014052 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:07.014101 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:07.014181 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:07.014670 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:07.014672 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:07.014673 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:07.014712 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:07.014713 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:07.016169 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:07.016170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:07.017104 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:07.017106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:07.018073 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:07.018074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:07.018137 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:07.018140 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:16:07.018168 (+ 28us) client_negotiation.cc:240] Negotiation successful
0924 21:16:07.018204 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":56,"threads_started":1}
I20260924 21:16:07.017999 24975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:07.013902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51524 (local address 127.6.210.129:39471)
0924 21:16:07.014370 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:07.014375 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:07.014386 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:16:07.014450 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:07.014452 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:07.014467 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:07.014753 (+ 286us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:07.014814 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:07.015499 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:07.016818 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:07.017543 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:07.017623 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:07.017719 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:07.017869 (+ 150us) server_negotiation.cc:299] Negotiation successful
0924 21:16:07.017910 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":99,"threads_started":1}
I20260924 21:16:07.018684 24976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:07.013934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59274 (local address 127.6.210.130:38563)
0924 21:16:07.014452 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:07.014456 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:07.014502 (+ 46us) server_negotiation.cc:407] Connection header received
0924 21:16:07.014549 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:07.014552 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:07.014568 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:07.014785 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:07.014844 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:07.016074 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:07.017188 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:07.017968 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:07.018414 (+ 446us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:07.018516 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:07.018541 (+ 25us) server_negotiation.cc:299] Negotiation successful
0924 21:16:07.018593 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":90,"threads_started":1}
I20260924 21:16:07.019011 24637 tablet_service.cc:1467] Processing CreateTablet for tablet 4fef73c90747402eb4a16df4678dc70c (DEFAULT_TABLE table=test-table [id=030e3e4c0d3e428f86b2db27ffa5345a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:07.019369 24973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:07.014005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:45751 (local address 127.0.0.1:52098)
0924 21:16:07.014134 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:07.014145 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:07.014196 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:07.014586 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:07.014587 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:07.014590 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:07.014670 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:07.014672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:07.017178 (+ 2506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:07.017179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:07.018023 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:07.018025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:07.019104 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:07.019105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:07.019165 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:07.019170 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 21:16:07.019254 (+ 84us) client_negotiation.cc:240] Negotiation successful
0924 21:16:07.019290 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":55,"threads_started":1}
I20260924 21:16:07.019404 24637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fef73c90747402eb4a16df4678dc70c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:07.019671 24978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:07.014189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52098 (local address 127.6.210.131:45751)
0924 21:16:07.014388 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:07.014392 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:07.014403 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:16:07.014446 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:07.014449 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:07.014467 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:07.014758 (+ 291us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:07.014818 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:07.016074 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:07.018136 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:07.018999 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:07.019104 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:07.019193 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:07.019534 (+ 341us) server_negotiation.cc:299] Negotiation successful
0924 21:16:07.019591 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":106,"threads_started":1}
I20260924 21:16:07.019883 24768 tablet_service.cc:1467] Processing CreateTablet for tablet 4fef73c90747402eb4a16df4678dc70c (DEFAULT_TABLE table=test-table [id=030e3e4c0d3e428f86b2db27ffa5345a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:07.020226 24768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fef73c90747402eb4a16df4678dc70c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:07.020342 24899 tablet_service.cc:1467] Processing CreateTablet for tablet 4fef73c90747402eb4a16df4678dc70c (DEFAULT_TABLE table=test-table [id=030e3e4c0d3e428f86b2db27ffa5345a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:07.020681 24899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fef73c90747402eb4a16df4678dc70c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:07.022933 24979 tablet_bootstrap.cc:492] T 4fef73c90747402eb4a16df4678dc70c P 085bbbf35613436ea3e80a7624832eb2: Bootstrap starting.
I20260924 21:16:07.023466 24980 tablet_bootstrap.cc:492] T 4fef73c90747402eb4a16df4678dc70c P 5da66b3225354b5b94b0a340dd17a362: Bootstrap starting.
I20260924 21:16:07.023646 24979 tablet_bootstrap.cc:654] T 4fef73c90747402eb4a16df4678dc70c P 085bbbf35613436ea3e80a7624832eb2: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:07.023989 24979 log.cc:824] T 4fef73c90747402eb4a16df4678dc70c P 085bbbf35613436ea3e80a7624832eb2: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:07.024147 24980 tablet_bootstrap.cc:654] T 4fef73c90747402eb4a16df4678dc70c P 5da66b3225354b5b94b0a340dd17a362: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:07.024446 24980 log.cc:824] T 4fef73c90747402eb4a16df4678dc70c P 5da66b3225354b5b94b0a340dd17a362: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:07.024786 24979 tablet_bootstrap.cc:492] T 4fef73c90747402eb4a16df4678dc70c P 085bbbf35613436ea3e80a7624832eb2: No bootstrap required, opened a new log
I20260924 21:16:07.024767 24981 tablet_bootstrap.cc:492] T 4fef73c90747402eb4a16df4678dc70c P fba7d076c379415f8264da94fd1f7601: Bootstrap starting.
I20260924 21:16:07.024909 24979 ts_tablet_manager.cc:1397] T 4fef73c90747402eb4a16df4678dc70c P 085bbbf35613436ea3e80a7624832eb2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:16:07.025141 24980 tablet_bootstrap.cc:492] T 4fef73c90747402eb4a16df4678dc70c P 5da66b3225354b5b94b0a340dd17a362: No bootstrap required, opened a new log
I20260924 21:16:07.025228 24980 ts_tablet_manager.cc:1397] T 4fef73c90747402eb4a16df4678dc70c P 5da66b3225354b5b94b0a340dd17a362: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:16:07.025389 24981 tablet_bootstrap.cc:654] T 4fef73c90747402eb4a16df4678dc70c P fba7d076c379415f8264da94fd1f7601: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:07.025776 24981 log.cc:824] T 4fef73c90747402eb4a16df4678dc70c P fba7d076c379415f8264da94fd1f7601: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:07.026619 24981 tablet_bootstrap.cc:492] T 4fef73c90747402eb4a16df4678dc70c P fba7d076c379415f8264da94fd1f7601: No bootstrap required, opened a new log
I20260924 21:16:07.026690 24981 ts_tablet_manager.cc:1397] T 4fef73c90747402eb4a16df4678dc70c P fba7d076c379415f8264da94fd1f7601: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:16:07.027415 24980 raft_consensus.cc:348] T 4fef73c90747402eb4a16df4678dc70c P 5da66b3225354b5b94b0a340dd17a362 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "085bbbf35613436ea3e80a7624832eb2" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39471 } } peers { permanent_uuid: "5da66b3225354b5b94b0a340dd17a362" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38563 } } peers { permanent_uuid: "fba7d076c379415f8264da94fd1f7601" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 45751 } }
I20260924 21:16:07.027585 24980 raft_consensus.cc:374] T 4fef73c90747402eb4a16df4678dc70c P 5da66b3225354b5b94b0a340dd17a362 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:07.027650 24980 raft_consensus.cc:729] T 4fef73c90747402eb4a16df4678dc70c P 5da66b3225354b5b94b0a340dd17a362 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5da66b3225354b5b94b0a340dd17a362, State: Initialized, Role: FOLLOWER
I20260924 21:16:07.027781 24980 consensus_queue.cc:260] T 4fef73c90747402eb4a16df4678dc70c P 5da66b3225354b5b94b0a340dd17a362 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "085bbbf35613436ea3e80a7624832eb2" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39471 } } peers { permanent_uuid: "5da66b3225354b5b94b0a340dd17a362" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38563 } } peers { permanent_uuid: "fba7d076c379415f8264da94fd1f7601" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 45751 } }
I20260924 21:16:07.028055 24980 ts_tablet_manager.cc:1428] T 4fef73c90747402eb4a16df4678dc70c P 5da66b3225354b5b94b0a340dd17a362: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:16:07.027945 24979 raft_consensus.cc:348] T 4fef73c90747402eb4a16df4678dc70c P 085bbbf35613436ea3e80a7624832eb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "085bbbf35613436ea3e80a7624832eb2" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39471 } } peers { permanent_uuid: "5da66b3225354b5b94b0a340dd17a362" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38563 } } peers { permanent_uuid: "fba7d076c379415f8264da94fd1f7601" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 45751 } }
I20260924 21:16:07.028118 24979 raft_consensus.cc:374] T 4fef73c90747402eb4a16df4678dc70c P 085bbbf35613436ea3e80a7624832eb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:07.028196 24834 heartbeater.cc:503] Master 127.6.210.190:36435 was elected leader, sending a full tablet report...
I20260924 21:16:07.028204 24979 raft_consensus.cc:729] T 4fef73c90747402eb4a16df4678dc70c P 085bbbf35613436ea3e80a7624832eb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 085bbbf35613436ea3e80a7624832eb2, State: Initialized, Role: FOLLOWER
I20260924 21:16:07.028318 24979 consensus_queue.cc:260] T 4fef73c90747402eb4a16df4678dc70c P 085bbbf35613436ea3e80a7624832eb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "085bbbf35613436ea3e80a7624832eb2" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39471 } } peers { permanent_uuid: "5da66b3225354b5b94b0a340dd17a362" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38563 } } peers { permanent_uuid: "fba7d076c379415f8264da94fd1f7601" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 45751 } }
I20260924 21:16:07.028569 24979 ts_tablet_manager.cc:1428] T 4fef73c90747402eb4a16df4678dc70c P 085bbbf35613436ea3e80a7624832eb2: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260924 21:16:07.028682 24703 heartbeater.cc:503] Master 127.6.210.190:36435 was elected leader, sending a full tablet report...
I20260924 21:16:07.029239 24981 raft_consensus.cc:348] T 4fef73c90747402eb4a16df4678dc70c P fba7d076c379415f8264da94fd1f7601 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "085bbbf35613436ea3e80a7624832eb2" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39471 } } peers { permanent_uuid: "5da66b3225354b5b94b0a340dd17a362" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38563 } } peers { permanent_uuid: "fba7d076c379415f8264da94fd1f7601" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 45751 } }
I20260924 21:16:07.029407 24981 raft_consensus.cc:374] T 4fef73c90747402eb4a16df4678dc70c P fba7d076c379415f8264da94fd1f7601 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:07.029443 24981 raft_consensus.cc:729] T 4fef73c90747402eb4a16df4678dc70c P fba7d076c379415f8264da94fd1f7601 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fba7d076c379415f8264da94fd1f7601, State: Initialized, Role: FOLLOWER
I20260924 21:16:07.029537 24981 consensus_queue.cc:260] T 4fef73c90747402eb4a16df4678dc70c P fba7d076c379415f8264da94fd1f7601 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "085bbbf35613436ea3e80a7624832eb2" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39471 } } peers { permanent_uuid: "5da66b3225354b5b94b0a340dd17a362" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38563 } } peers { permanent_uuid: "fba7d076c379415f8264da94fd1f7601" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 45751 } }
I20260924 21:16:07.029788 24981 ts_tablet_manager.cc:1428] T 4fef73c90747402eb4a16df4678dc70c P fba7d076c379415f8264da94fd1f7601: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:16:07.029819 24965 heartbeater.cc:503] Master 127.6.210.190:36435 was elected leader, sending a full tablet report...
I20260924 21:16:07.090409 24985 raft_consensus.cc:482] T 4fef73c90747402eb4a16df4678dc70c P 5da66b3225354b5b94b0a340dd17a362 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:16:07.090577 24985 raft_consensus.cc:504] T 4fef73c90747402eb4a16df4678dc70c P 5da66b3225354b5b94b0a340dd17a362 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "085bbbf35613436ea3e80a7624832eb2" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39471 } } peers { permanent_uuid: "5da66b3225354b5b94b0a340dd17a362" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38563 } } peers { permanent_uuid: "fba7d076c379415f8264da94fd1f7601" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 45751 } }
I20260924 21:16:07.090955 24985 leader_election.cc:288] T 4fef73c90747402eb4a16df4678dc70c P 5da66b3225354b5b94b0a340dd17a362 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 085bbbf35613436ea3e80a7624832eb2 (127.6.210.129:39471), fba7d076c379415f8264da94fd1f7601 (127.6.210.131:45751)
I20260924 21:16:07.093974 24978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:07.091244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:39487 (local address 127.6.210.131:45751)
0924 21:16:07.091289 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:07.091291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:07.091304 (+ 13us) server_negotiation.cc:407] Connection header received
0924 21:16:07.091339 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:07.091340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:07.091351 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:07.091435 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:07.091690 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:07.092243 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:07.093286 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:07.093627 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:07.093684 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:07.093690 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:07.093847 (+ 157us) server_negotiation.cc:299] Negotiation successful
0924 21:16:07.093894 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 21:16:07.094019 24988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:07.091170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:39471 (local address 127.6.210.130:40629)
0924 21:16:07.091360 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:07.091371 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:07.091430 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:07.091664 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:07.091665 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:07.091667 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:07.091739 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:07.091740 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:07.092359 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:07.092360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:07.093158 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:07.093160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:07.093747 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:07.093747 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:07.093807 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:07.093828 (+ 21us) client_negotiation.cc:769] Sending connection context
0924 21:16:07.093885 (+ 57us) client_negotiation.cc:240] Negotiation successful
0924 21:16:07.093920 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":38,"thread_start_us":118,"threads_started":1}
I20260924 21:16:07.094026 24975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:07.091244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:40629 (local address 127.6.210.129:39471)
0924 21:16:07.091289 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:07.091291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:07.091508 (+ 217us) server_negotiation.cc:407] Connection header received
0924 21:16:07.091546 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:07.091547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:07.091561 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:07.091639 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:07.091815 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:07.092259 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:07.093258 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:07.093639 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:07.093730 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:07.093738 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:07.093910 (+ 172us) server_negotiation.cc:299] Negotiation successful
0924 21:16:07.093955 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 21:16:07.094026 24836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:07.091169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:45751 (local address 127.6.210.130:39487)
0924 21:16:07.091226 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:07.091238 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:07.091283 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:07.091521 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:07.091523 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:07.091525 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:07.091613 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:07.091616 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:07.092327 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:07.092328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:07.093172 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:07.093174 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:07.093706 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:07.093708 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:07.093778 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:07.093781 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:16:07.093838 (+ 57us) client_negotiation.cc:240] Negotiation successful
0924 21:16:07.093897 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":3}
I20260924 21:16:07.094467 24919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fef73c90747402eb4a16df4678dc70c" candidate_uuid: "5da66b3225354b5b94b0a340dd17a362" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fba7d076c379415f8264da94fd1f7601" is_pre_election: true
I20260924 21:16:07.094456 24657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fef73c90747402eb4a16df4678dc70c" candidate_uuid: "5da66b3225354b5b94b0a340dd17a362" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "085bbbf35613436ea3e80a7624832eb2" is_pre_election: true
I20260924 21:16:07.094640 24919 raft_consensus.cc:2454] T 4fef73c90747402eb4a16df4678dc70c P fba7d076c379415f8264da94fd1f7601 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5da66b3225354b5b94b0a340dd17a362 in term 0.
I20260924 21:16:07.094652 24657 raft_consensus.cc:2454] T 4fef73c90747402eb4a16df4678dc70c P 085bbbf35613436ea3e80a7624832eb2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5da66b3225354b5b94b0a340dd17a362 in term 0.
I20260924 21:16:07.094892 24721 leader_election.cc:302] T 4fef73c90747402eb4a16df4678dc70c P 5da66b3225354b5b94b0a340dd17a362 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 085bbbf35613436ea3e80a7624832eb2, 5da66b3225354b5b94b0a340dd17a362; no voters:
I20260924 21:16:07.095053 24985 raft_consensus.cc:2781] T 4fef73c90747402eb4a16df4678dc70c P 5da66b3225354b5b94b0a340dd17a362 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:16:07.095109 24985 raft_consensus.cc:482] T 4fef73c90747402eb4a16df4678dc70c P 5da66b3225354b5b94b0a340dd17a362 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:16:07.095134 24985 raft_consensus.cc:3037] T 4fef73c90747402eb4a16df4678dc70c P 5da66b3225354b5b94b0a340dd17a362 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:07.095781 24985 raft_consensus.cc:504] T 4fef73c90747402eb4a16df4678dc70c P 5da66b3225354b5b94b0a340dd17a362 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "085bbbf35613436ea3e80a7624832eb2" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39471 } } peers { permanent_uuid: "5da66b3225354b5b94b0a340dd17a362" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38563 } } peers { permanent_uuid: "fba7d076c379415f8264da94fd1f7601" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 45751 } }
I20260924 21:16:07.095924 24985 leader_election.cc:288] T 4fef73c90747402eb4a16df4678dc70c P 5da66b3225354b5b94b0a340dd17a362 [CANDIDATE]: Term 1 election: Requested vote from peers 085bbbf35613436ea3e80a7624832eb2 (127.6.210.129:39471), fba7d076c379415f8264da94fd1f7601 (127.6.210.131:45751)
I20260924 21:16:07.096073 24919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fef73c90747402eb4a16df4678dc70c" candidate_uuid: "5da66b3225354b5b94b0a340dd17a362" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fba7d076c379415f8264da94fd1f7601"
I20260924 21:16:07.096104 24657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fef73c90747402eb4a16df4678dc70c" candidate_uuid: "5da66b3225354b5b94b0a340dd17a362" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "085bbbf35613436ea3e80a7624832eb2"
I20260924 21:16:07.096148 24919 raft_consensus.cc:3037] T 4fef73c90747402eb4a16df4678dc70c P fba7d076c379415f8264da94fd1f7601 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:07.096212 24657 raft_consensus.cc:3037] T 4fef73c90747402eb4a16df4678dc70c P 085bbbf35613436ea3e80a7624832eb2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:07.096932 24657 raft_consensus.cc:2454] T 4fef73c90747402eb4a16df4678dc70c P 085bbbf35613436ea3e80a7624832eb2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5da66b3225354b5b94b0a340dd17a362 in term 1.
I20260924 21:16:07.096935 24919 raft_consensus.cc:2454] T 4fef73c90747402eb4a16df4678dc70c P fba7d076c379415f8264da94fd1f7601 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5da66b3225354b5b94b0a340dd17a362 in term 1.
I20260924 21:16:07.097098 24721 leader_election.cc:302] T 4fef73c90747402eb4a16df4678dc70c P 5da66b3225354b5b94b0a340dd17a362 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 085bbbf35613436ea3e80a7624832eb2, 5da66b3225354b5b94b0a340dd17a362; no voters:
I20260924 21:16:07.097199 24985 raft_consensus.cc:2781] T 4fef73c90747402eb4a16df4678dc70c P 5da66b3225354b5b94b0a340dd17a362 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:07.097349 24985 raft_consensus.cc:686] T 4fef73c90747402eb4a16df4678dc70c P 5da66b3225354b5b94b0a340dd17a362 [term 1 LEADER]: Becoming Leader. State: Replica: 5da66b3225354b5b94b0a340dd17a362, State: Running, Role: LEADER
I20260924 21:16:07.097455 24985 consensus_queue.cc:237] T 4fef73c90747402eb4a16df4678dc70c P 5da66b3225354b5b94b0a340dd17a362 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "085bbbf35613436ea3e80a7624832eb2" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39471 } } peers { permanent_uuid: "5da66b3225354b5b94b0a340dd17a362" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38563 } } peers { permanent_uuid: "fba7d076c379415f8264da94fd1f7601" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 45751 } }
I20260924 21:16:07.098377 24514 catalog_manager.cc:5495] T 4fef73c90747402eb4a16df4678dc70c P 5da66b3225354b5b94b0a340dd17a362 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5da66b3225354b5b94b0a340dd17a362 (127.6.210.130). New cstate: current_term: 1 leader_uuid: "5da66b3225354b5b94b0a340dd17a362" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "085bbbf35613436ea3e80a7624832eb2" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5da66b3225354b5b94b0a340dd17a362" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38563 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fba7d076c379415f8264da94fd1f7601" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 45751 } health_report { overall_health: UNKNOWN } } }
I20260924 21:16:07.109983 24976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:07.107551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59290 (local address 127.6.210.130:38563)
0924 21:16:07.107616 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:07.107618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:07.107655 (+ 37us) server_negotiation.cc:407] Connection header received
0924 21:16:07.107697 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:07.107698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:07.107718 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:07.107810 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:16:07.108005 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:07.108612 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:07.109358 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:07.109544 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:07.109600 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:07.109719 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:16:07.109810 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:16:07.109864 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:07.109899 (+ 35us) server_negotiation.cc:299] Negotiation successful
0924 21:16:07.109913 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260924 21:16:07.111945 24657 raft_consensus.cc:1261] T 4fef73c90747402eb4a16df4678dc70c P 085bbbf35613436ea3e80a7624832eb2 [term 1 FOLLOWER]: Refusing update from remote peer 5da66b3225354b5b94b0a340dd17a362: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:16:07.111961 24919 raft_consensus.cc:1261] T 4fef73c90747402eb4a16df4678dc70c P fba7d076c379415f8264da94fd1f7601 [term 1 FOLLOWER]: Refusing update from remote peer 5da66b3225354b5b94b0a340dd17a362: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:16:07.112409 24989 consensus_queue.cc:1035] T 4fef73c90747402eb4a16df4678dc70c P 5da66b3225354b5b94b0a340dd17a362 [LEADER]: Connected to new peer: Peer: permanent_uuid: "085bbbf35613436ea3e80a7624832eb2" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 39471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:16:07.112571 24985 consensus_queue.cc:1035] T 4fef73c90747402eb4a16df4678dc70c P 5da66b3225354b5b94b0a340dd17a362 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fba7d076c379415f8264da94fd1f7601" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 45751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:16:07.115677 24994 mvcc.cc:205] Tried to move back new op lower bound from 7333005586887536640 to 7333005586831921152. Current Snapshot: MvccSnapshot[applied={T|T < 7333005586887536640}]
I20260924 21:16:07.117355 24514 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:16:07.117452 24514 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:16:07.118295 24514 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d521507173a7425a92d559a59624a42c: Sending DeleteTablet for 3 replicas of tablet 4fef73c90747402eb4a16df4678dc70c
I20260924 21:16:07.118671 24899 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fef73c90747402eb4a16df4678dc70c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:07 UTC) from {username='slave'} at 127.0.0.1:52098
I20260924 21:16:07.118674 24637 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fef73c90747402eb4a16df4678dc70c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:07 UTC) from {username='slave'} at 127.0.0.1:51524
I20260924 21:16:07.118927 24768 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fef73c90747402eb4a16df4678dc70c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:07 UTC) from {username='slave'} at 127.0.0.1:59274
I20260924 21:16:07.118983 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 24575
I20260924 21:16:07.119994 25005 tablet_replica.cc:330] T 4fef73c90747402eb4a16df4678dc70c P 5da66b3225354b5b94b0a340dd17a362: stopping tablet replica
I20260924 21:16:07.120105 25005 raft_consensus.cc:2229] T 4fef73c90747402eb4a16df4678dc70c P 5da66b3225354b5b94b0a340dd17a362 [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:16:07.120195 25005 raft_consensus.cc:2258] T 4fef73c90747402eb4a16df4678dc70c P 5da66b3225354b5b94b0a340dd17a362 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:16:07.120468 25004 tablet_replica.cc:330] T 4fef73c90747402eb4a16df4678dc70c P fba7d076c379415f8264da94fd1f7601: stopping tablet replica
I20260924 21:16:07.120585 25004 raft_consensus.cc:2229] T 4fef73c90747402eb4a16df4678dc70c P fba7d076c379415f8264da94fd1f7601 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:16:07.120683 25004 raft_consensus.cc:2258] T 4fef73c90747402eb4a16df4678dc70c P fba7d076c379415f8264da94fd1f7601 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:16:07.121074 25004 ts_tablet_manager.cc:1905] T 4fef73c90747402eb4a16df4678dc70c P fba7d076c379415f8264da94fd1f7601: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:16:07.122699 25004 ts_tablet_manager.cc:1918] T 4fef73c90747402eb4a16df4678dc70c P fba7d076c379415f8264da94fd1f7601: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:16:07.122788 25004 log.cc:1196] T 4fef73c90747402eb4a16df4678dc70c P fba7d076c379415f8264da94fd1f7601: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-2/wal/wals/4fef73c90747402eb4a16df4678dc70c
I20260924 21:16:07.123024 25004 ts_tablet_manager.cc:1939] T 4fef73c90747402eb4a16df4678dc70c P fba7d076c379415f8264da94fd1f7601: Deleting consensus metadata
I20260924 21:16:07.123414 24502 catalog_manager.cc:4841] TS fba7d076c379415f8264da94fd1f7601 (127.6.210.131:45751): tablet 4fef73c90747402eb4a16df4678dc70c (table test-table [id=030e3e4c0d3e428f86b2db27ffa5345a]) successfully deleted
I20260924 21:16:07.124125 25005 ts_tablet_manager.cc:1905] T 4fef73c90747402eb4a16df4678dc70c P 5da66b3225354b5b94b0a340dd17a362: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:16:07.125485 25005 ts_tablet_manager.cc:1918] T 4fef73c90747402eb4a16df4678dc70c P 5da66b3225354b5b94b0a340dd17a362: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:16:07.125578 25005 log.cc:1196] T 4fef73c90747402eb4a16df4678dc70c P 5da66b3225354b5b94b0a340dd17a362: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284333163253-6986-0/minicluster-data/ts-1/wal/wals/4fef73c90747402eb4a16df4678dc70c
W20260924 21:16:07.125808 24499 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.210.129:39471 (error 108)
I20260924 21:16:07.125845 25005 ts_tablet_manager.cc:1939] T 4fef73c90747402eb4a16df4678dc70c P 5da66b3225354b5b94b0a340dd17a362: Deleting consensus metadata
I20260924 21:16:07.126082 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 24706
I20260924 21:16:07.128646 24973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:07.128448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:39471 (local address 127.0.0.1:51534)
0924 21:16:07.128549 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:07.128605 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.129:39471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260924 21:16:07.128798 24499 catalog_manager.cc:4584] TS 085bbbf35613436ea3e80a7624832eb2 (127.6.210.129:39471): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4fef73c90747402eb4a16df4678dc70c: Network error: Client connection negotiation failed: client connection to 127.6.210.129:39471: connect: Connection refused (error 111)
I20260924 21:16:07.131958 24973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:07.131839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:38563 (local address 127.0.0.1:59304)
0924 21:16:07.131909 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:07.131932 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.130:38563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260924 21:16:07.131960 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 24837
I20260924 21:16:07.136447 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 24484
2026-09-24T21:16:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (725 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:16:07 dist-test-slave-q276 krb5kdc[25009](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:16:07 dist-test-slave-q276 krb5kdc[25009](info): set up 2 sockets
Sep 24 21:16:07 dist-test-slave-q276 krb5kdc[25009](info): commencing operation
krb5kdc: starting...
W20260924 21:16:08.166153 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:16:08 dist-test-slave-q276 krb5kdc[25009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:16:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:16:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.210.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
I20260924 21:16:08.287123 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:33135
--webserver_interface=127.6.210.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:33337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:33135
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:08.374157 25022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:08.374413 25022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:08.374442 25022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:08.376108 25022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:08.376164 25022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:08.376181 25022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:08.376202 25022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:08.376215 25022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:08.377830 25022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:33337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:33135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:33135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:08.378096 25022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:08.378378 25022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:08.384338 25030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:08.384370 25027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:08.384497 25028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:08.384411 25022 server_base.cc:1034] running on GCE node
I20260924 21:16:08.384752 25022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:08.385010 25022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:08.386164 25022 hybrid_clock.cc:648] HybridClock initialized: now 1790284568386153 us; error 34 us; skew 500 ppm
Sep 24 21:16:08 dist-test-slave-q276 krb5kdc[25009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284568, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:08.388715 25022 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:16:08.389129 25022 webserver.cc:460] Webserver started at http://127.6.210.190:40885/ using document root <none> and password file <none>
I20260924 21:16:08.389412 25022 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:08.389472 25022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:08.389571 25022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:08.390686 25022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "41e8e56b81d0467f8fc036b353c3e3a2"
format_stamp: "Formatted at 2026-09-24 21:16:08 on dist-test-slave-q276"
server_key: "30c51d57c3ac41c5752194490ea3816c"
server_key_iv: "ae897b6dfed785fe14ae45a2324b8d73"
server_key_version: "encryptionkey@0"
I20260924 21:16:08.390992 25022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "41e8e56b81d0467f8fc036b353c3e3a2"
format_stamp: "Formatted at 2026-09-24 21:16:08 on dist-test-slave-q276"
server_key: "30c51d57c3ac41c5752194490ea3816c"
server_key_iv: "ae897b6dfed785fe14ae45a2324b8d73"
server_key_version: "encryptionkey@0"
I20260924 21:16:08.392242 25022 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:16:08.393016 25037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:08.393242 25022 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:08.393324 25022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "41e8e56b81d0467f8fc036b353c3e3a2"
format_stamp: "Formatted at 2026-09-24 21:16:08 on dist-test-slave-q276"
server_key: "30c51d57c3ac41c5752194490ea3816c"
server_key_iv: "ae897b6dfed785fe14ae45a2324b8d73"
server_key_version: "encryptionkey@0"
I20260924 21:16:08.393419 25022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:08.404330 25022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:08.406689 25022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:08.406800 25022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:08.410651 25022 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:33135
I20260924 21:16:08.410727 25089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:33135 every 8 connection(s)
I20260924 21:16:08.411082 25022 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:16:08.411969 25090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:08.412034 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 25022
I20260924 21:16:08.412168 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:16:08.412362 6986 external_mini_cluster.cc:1428] Setting key 1aef377de9866bef5f0bbe632489ab46
I20260924 21:16:08.415333 25090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41e8e56b81d0467f8fc036b353c3e3a2: Bootstrap starting.
I20260924 21:16:08.416015 25090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41e8e56b81d0467f8fc036b353c3e3a2: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:08.416317 25090 log.cc:824] T 00000000000000000000000000000000 P 41e8e56b81d0467f8fc036b353c3e3a2: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:08.417055 25090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41e8e56b81d0467f8fc036b353c3e3a2: No bootstrap required, opened a new log
Sep 24 21:16:08 dist-test-slave-q276 krb5kdc[25009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:16:08.419705 25090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41e8e56b81d0467f8fc036b353c3e3a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41e8e56b81d0467f8fc036b353c3e3a2" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 33135 } }
I20260924 21:16:08.419879 25090 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41e8e56b81d0467f8fc036b353c3e3a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:08.419927 25090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41e8e56b81d0467f8fc036b353c3e3a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41e8e56b81d0467f8fc036b353c3e3a2, State: Initialized, Role: FOLLOWER
I20260924 21:16:08.420136 25090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41e8e56b81d0467f8fc036b353c3e3a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41e8e56b81d0467f8fc036b353c3e3a2" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 33135 } }
I20260924 21:16:08.420256 25095 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41e8e56b81d0467f8fc036b353c3e3a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:16:08.420318 25095 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41e8e56b81d0467f8fc036b353c3e3a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41e8e56b81d0467f8fc036b353c3e3a2" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 33135 } }
I20260924 21:16:08.420440 25095 leader_election.cc:302] T 00000000000000000000000000000000 P 41e8e56b81d0467f8fc036b353c3e3a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41e8e56b81d0467f8fc036b353c3e3a2; no voters:
I20260924 21:16:08.420236 25090 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41e8e56b81d0467f8fc036b353c3e3a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:16:08.420531 25090 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41e8e56b81d0467f8fc036b353c3e3a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:16:08.420574 25090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41e8e56b81d0467f8fc036b353c3e3a2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:08.420603 25095 leader_election.cc:288] T 00000000000000000000000000000000 P 41e8e56b81d0467f8fc036b353c3e3a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260924 21:16:08.421164 25090 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41e8e56b81d0467f8fc036b353c3e3a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41e8e56b81d0467f8fc036b353c3e3a2" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 33135 } }
I20260924 21:16:08.421257 25090 leader_election.cc:302] T 00000000000000000000000000000000 P 41e8e56b81d0467f8fc036b353c3e3a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41e8e56b81d0467f8fc036b353c3e3a2; no voters:
I20260924 21:16:08.421273 25095 raft_consensus.cc:2741] T 00000000000000000000000000000000 P 41e8e56b81d0467f8fc036b353c3e3a2 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260924 21:16:08.421305 25090 leader_election.cc:288] T 00000000000000000000000000000000 P 41e8e56b81d0467f8fc036b353c3e3a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:16:08.421368 25095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41e8e56b81d0467f8fc036b353c3e3a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:08.421411 25095 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41e8e56b81d0467f8fc036b353c3e3a2 [term 1 LEADER]: Becoming Leader. State: Replica: 41e8e56b81d0467f8fc036b353c3e3a2, State: Running, Role: LEADER
I20260924 21:16:08.421469 25095 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41e8e56b81d0467f8fc036b353c3e3a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41e8e56b81d0467f8fc036b353c3e3a2" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 33135 } }
I20260924 21:16:08.421679 25090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41e8e56b81d0467f8fc036b353c3e3a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:08.421788 25095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41e8e56b81d0467f8fc036b353c3e3a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41e8e56b81d0467f8fc036b353c3e3a2. Latest consensus state: current_term: 1 leader_uuid: "41e8e56b81d0467f8fc036b353c3e3a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41e8e56b81d0467f8fc036b353c3e3a2" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 33135 } } }
I20260924 21:16:08.421866 25095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41e8e56b81d0467f8fc036b353c3e3a2 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:08.423476 25096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41e8e56b81d0467f8fc036b353c3e3a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41e8e56b81d0467f8fc036b353c3e3a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41e8e56b81d0467f8fc036b353c3e3a2" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 33135 } } }
I20260924 21:16:08.423576 25096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41e8e56b81d0467f8fc036b353c3e3a2 [sys.catalog]: This master's current role is: LEADER
W20260924 21:16:08.423722 25111 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 41e8e56b81d0467f8fc036b353c3e3a2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 21:16:08.423823 25111 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 21:16:08.423964 25099 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:16:08.424217 25099 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:16:08.424795 25093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:08.412986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56324 (local address 127.6.210.190:33135)
0924 21:16:08.413185 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:08.413194 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:08.413220 (+ 26us) server_negotiation.cc:407] Connection header received
0924 21:16:08.413672 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:08.413679 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:08.413728 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:08.413853 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:08.414738 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:08.415764 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:08.416785 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:08.417283 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:08.417371 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:08.420238 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:08.420243 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:08.420244 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:08.420258 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:08.422947 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:08.423431 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:08.423434 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:08.423434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:08.423541 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:08.423800 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:08.423800 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:08.423801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:08.424132 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:08.424314 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:08.424395 (+ 81us) server_negotiation.cc:299] Negotiation successful
0924 21:16:08.424539 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":100,"threads_started":1}
I20260924 21:16:08.425906 25099 catalog_manager.cc:1347] Generated new cluster ID: 155dc711407545c08e34d1a6c7c7c9e2
I20260924 21:16:08.425966 25099 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:16:08.440676 25099 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:16:08.441241 25099 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:16:08.446817 25099 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41e8e56b81d0467f8fc036b353c3e3a2: Generated new TSK 0
I20260924 21:16:08.447041 25099 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.210.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
I20260924 21:16:08.482008 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.129:0
--local_ip_for_outbound_sockets=127.6.210.129
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:33135
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:33337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:08.568092 25116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:08.568337 25116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:08.568377 25116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:08.569542 25116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:08.569613 25116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:08.569657 25116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.129
I20260924 21:16:08.571254 25116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:33337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:33135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.129
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:08.571538 25116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:08.571770 25116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:08.576972 25121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:08.576977 25122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:08.577050 25116 server_base.cc:1034] running on GCE node
W20260924 21:16:08.577289 25124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:08.577495 25116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:08.577726 25116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:08.578876 25116 hybrid_clock.cc:648] HybridClock initialized: now 1790284568578848 us; error 43 us; skew 500 ppm
Sep 24 21:16:08 dist-test-slave-q276 krb5kdc[25009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284568, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:08.588804 25116 init.cc:377] Logged in from keytab as kudu/127.6.210.129@KRBTEST.COM (short username kudu)
I20260924 21:16:08.589272 25116 webserver.cc:460] Webserver started at http://127.6.210.129:36475/ using document root <none> and password file <none>
I20260924 21:16:08.589529 25116 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:08.589602 25116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:08.589779 25116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:08.590922 25116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-0/data/instance:
uuid: "36c458ef6166447380fbce070938173a"
format_stamp: "Formatted at 2026-09-24 21:16:08 on dist-test-slave-q276"
server_key: "2c8cd117a7c5b1dfee9fb702aba9a6e8"
server_key_iv: "30f023e12e547a0554eceb64381b25ea"
server_key_version: "encryptionkey@0"
I20260924 21:16:08.591279 25116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance:
uuid: "36c458ef6166447380fbce070938173a"
format_stamp: "Formatted at 2026-09-24 21:16:08 on dist-test-slave-q276"
server_key: "2c8cd117a7c5b1dfee9fb702aba9a6e8"
server_key_iv: "30f023e12e547a0554eceb64381b25ea"
server_key_version: "encryptionkey@0"
I20260924 21:16:08.592664 25116 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:16:08.593472 25131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:08.593683 25116 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:08.593778 25116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-0/wal
uuid: "36c458ef6166447380fbce070938173a"
format_stamp: "Formatted at 2026-09-24 21:16:08 on dist-test-slave-q276"
server_key: "2c8cd117a7c5b1dfee9fb702aba9a6e8"
server_key_iv: "30f023e12e547a0554eceb64381b25ea"
server_key_version: "encryptionkey@0"
I20260924 21:16:08.593833 25116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:08.620944 25116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:08.623617 25116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:08.623741 25116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:08.623934 25116 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:16:08.624264 25116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:16:08.624305 25116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:08.624329 25116 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:16:08.624378 25116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:08.631673 25116 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.129:33059
I20260924 21:16:08.631753 25244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.129:33059 every 8 connection(s)
I20260924 21:16:08.632114 25116 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
I20260924 21:16:08.637905 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 25116
I20260924 21:16:08.638038 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance
I20260924 21:16:08.638235 6986 external_mini_cluster.cc:1428] Setting key 06a6fb3d8def9bf5c4b59d2881838cc2
Sep 24 21:16:08 dist-test-slave-q276 krb5kdc[25009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284568, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:16:08.642231 25093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:08.632900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:37285 (local address 127.6.210.190:33135)
0924 21:16:08.632984 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:08.632987 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:08.633157 (+ 170us) server_negotiation.cc:407] Connection header received
0924 21:16:08.633634 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:08.633635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:08.633646 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:08.633704 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:08.634704 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:08.635211 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:08.636192 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:08.636395 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:08.636447 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:08.638960 (+ 2513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:08.638961 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:08.638962 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:08.638968 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:08.640670 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:08.641189 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:08.641190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:08.641191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:08.641247 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:08.641547 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:08.641548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:08.641548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:08.641660 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:08.641713 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:08.642081 (+ 368us) server_negotiation.cc:299] Negotiation successful
0924 21:16:08.642162 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260924 21:16:08.642511 25247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:08.632863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:33135 (local address 127.6.210.129:37285)
0924 21:16:08.633026 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:08.633063 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:08.633531 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:08.633768 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:08.633771 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:08.634399 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:08.634607 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:08.634612 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:08.635294 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:08.635295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:08.636091 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:08.636093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:08.636472 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:08.636472 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:08.636582 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:08.637207 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:08.637216 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:08.638833 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:08.640753 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:08.640760 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:08.640763 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:08.641082 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:08.641335 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:08.641336 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:08.641337 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:08.641480 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:08.641727 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:08.641733 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:08.641901 (+ 168us) client_negotiation.cc:769] Sending connection context
0924 21:16:08.642019 (+ 118us) client_negotiation.cc:240] Negotiation successful
0924 21:16:08.642200 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":99,"threads_started":1}
I20260924 21:16:08.643059 25245 heartbeater.cc:347] Connected to a master server at 127.6.210.190:33135
I20260924 21:16:08.643185 25245 heartbeater.cc:464] Registering TS with master...
I20260924 21:16:08.643393 25245 heartbeater.cc:511] Master 127.6.210.190:33135 requested a full tablet report, sending...
I20260924 21:16:08.644039 25054 ts_manager.cc:194] Registered new tserver with Master: 36c458ef6166447380fbce070938173a (127.6.210.129:33059)
I20260924 21:16:08.644750 25054 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.129@KRBTEST.COM'} at 127.6.210.129:37285
WARNING: no policy specified for kudu/127.6.210.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
I20260924 21:16:08.664716 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.130:0
--local_ip_for_outbound_sockets=127.6.210.130
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:33135
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:33337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:08.748304 25250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:08.748566 25250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:08.748592 25250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:08.749866 25250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:08.749917 25250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:08.749953 25250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.130
I20260924 21:16:08.751489 25250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:33337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:33135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.130
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:08.751727 25250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:08.751999 25250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:08.757639 25258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:08.757644 25255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:08.757745 25256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:08.757689 25250 server_base.cc:1034] running on GCE node
I20260924 21:16:08.758044 25250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:08.758319 25250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:08.759469 25250 hybrid_clock.cc:648] HybridClock initialized: now 1790284568759457 us; error 28 us; skew 500 ppm
Sep 24 21:16:08 dist-test-slave-q276 krb5kdc[25009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284568, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:08.761940 25250 init.cc:377] Logged in from keytab as kudu/127.6.210.130@KRBTEST.COM (short username kudu)
I20260924 21:16:08.762416 25250 webserver.cc:460] Webserver started at http://127.6.210.130:45623/ using document root <none> and password file <none>
I20260924 21:16:08.762663 25250 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:08.762741 25250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:08.762902 25250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:08.764000 25250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-1/data/instance:
uuid: "377952d106c743958279dfe30271e4b0"
format_stamp: "Formatted at 2026-09-24 21:16:08 on dist-test-slave-q276"
server_key: "79d2fe5daf3c49abb9f15d5c7442f0f9"
server_key_iv: "9df3e02996e4eb5081a0c5c6ff74e632"
server_key_version: "encryptionkey@0"
I20260924 21:16:08.764499 25250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance:
uuid: "377952d106c743958279dfe30271e4b0"
format_stamp: "Formatted at 2026-09-24 21:16:08 on dist-test-slave-q276"
server_key: "79d2fe5daf3c49abb9f15d5c7442f0f9"
server_key_iv: "9df3e02996e4eb5081a0c5c6ff74e632"
server_key_version: "encryptionkey@0"
I20260924 21:16:08.765901 25250 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:16:08.766813 25265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:08.766981 25250 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:08.767153 25250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-1/wal
uuid: "377952d106c743958279dfe30271e4b0"
format_stamp: "Formatted at 2026-09-24 21:16:08 on dist-test-slave-q276"
server_key: "79d2fe5daf3c49abb9f15d5c7442f0f9"
server_key_iv: "9df3e02996e4eb5081a0c5c6ff74e632"
server_key_version: "encryptionkey@0"
I20260924 21:16:08.767239 25250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:08.778144 25250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:08.780542 25250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:08.780718 25250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:08.781018 25250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:16:08.781302 25250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:16:08.781337 25250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:08.781360 25250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:16:08.781375 25250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:08.787498 25250 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.130:43321
I20260924 21:16:08.787536 25378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.130:43321 every 8 connection(s)
I20260924 21:16:08.787883 25250 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
I20260924 21:16:08.789598 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 25250
I20260924 21:16:08.789728 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance
I20260924 21:16:08.789973 6986 external_mini_cluster.cc:1428] Setting key 53f8d4778516638193db77765e68dad3
Sep 24 21:16:08 dist-test-slave-q276 krb5kdc[25009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284568, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:16:08.799242 25093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:08.788599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:54709 (local address 127.6.210.190:33135)
0924 21:16:08.788669 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:08.788672 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:08.788914 (+ 242us) server_negotiation.cc:407] Connection header received
0924 21:16:08.789524 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:08.789526 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:08.789538 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:08.789599 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:08.790957 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:08.791754 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:08.792937 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:08.793174 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:08.793269 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:08.795880 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:08.795882 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:08.795883 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:08.795893 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:08.797753 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:08.798175 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:08.798177 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:08.798178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:08.798248 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:08.798523 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:08.798524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:08.798525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:08.798712 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:08.798794 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:08.799044 (+ 250us) server_negotiation.cc:299] Negotiation successful
0924 21:16:08.799138 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 21:16:08.799278 25381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:08.788592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:33135 (local address 127.6.210.130:54709)
0924 21:16:08.788786 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:08.788823 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:08.789386 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:08.789702 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:08.789707 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:08.790558 (+ 851us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:08.790859 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:08.790865 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:08.791847 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:08.791848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:08.792810 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:08.792813 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:08.793282 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:08.793283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:08.793442 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:08.793953 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:08.793964 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:08.795786 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:08.797848 (+ 2062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:08.797853 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:08.797855 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:08.798074 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:08.798342 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:08.798343 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:08.798344 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:08.798449 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:08.798787 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:08.798791 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:08.798907 (+ 116us) client_negotiation.cc:769] Sending connection context
0924 21:16:08.798990 (+ 83us) client_negotiation.cc:240] Negotiation successful
0924 21:16:08.799113 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":88,"threads_started":1}
I20260924 21:16:08.799852 25379 heartbeater.cc:347] Connected to a master server at 127.6.210.190:33135
I20260924 21:16:08.799942 25379 heartbeater.cc:464] Registering TS with master...
I20260924 21:16:08.800123 25379 heartbeater.cc:511] Master 127.6.210.190:33135 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.6.210.131@KRBTEST.COM; defaulting to no policy
I20260924 21:16:08.800491 25054 ts_manager.cc:194] Registered new tserver with Master: 377952d106c743958279dfe30271e4b0 (127.6.210.130:43321)
I20260924 21:16:08.800900 25054 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.130@KRBTEST.COM'} at 127.6.210.130:54709
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
I20260924 21:16:08.821214 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.131:0
--local_ip_for_outbound_sockets=127.6.210.131
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:33135
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:33337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:08.903642 25384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:08.903880 25384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:08.903919 25384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:08.905066 25384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:08.905133 25384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:08.905179 25384 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.131
I20260924 21:16:08.906632 25384 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:33337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:33135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.131
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:08.906893 25384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:08.907135 25384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:08.912047 25392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:08.912110 25389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:08.912115 25390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:08.912235 25384 server_base.cc:1034] running on GCE node
I20260924 21:16:08.912451 25384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:08.912712 25384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:08.913859 25384 hybrid_clock.cc:648] HybridClock initialized: now 1790284568913849 us; error 25 us; skew 500 ppm
Sep 24 21:16:08 dist-test-slave-q276 krb5kdc[25009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284568, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:08.916316 25384 init.cc:377] Logged in from keytab as kudu/127.6.210.131@KRBTEST.COM (short username kudu)
I20260924 21:16:08.916800 25384 webserver.cc:460] Webserver started at http://127.6.210.131:42635/ using document root <none> and password file <none>
I20260924 21:16:08.917026 25384 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:08.917096 25384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:08.917248 25384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:08.918336 25384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-2/data/instance:
uuid: "870409d09d654995b8b05d099f9fc7dc"
format_stamp: "Formatted at 2026-09-24 21:16:08 on dist-test-slave-q276"
server_key: "7d34c1979f78d7fbe974747801b8b572"
server_key_iv: "cb558f5b06711ac6ff6693527876387e"
server_key_version: "encryptionkey@0"
I20260924 21:16:08.918726 25384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance:
uuid: "870409d09d654995b8b05d099f9fc7dc"
format_stamp: "Formatted at 2026-09-24 21:16:08 on dist-test-slave-q276"
server_key: "7d34c1979f78d7fbe974747801b8b572"
server_key_iv: "cb558f5b06711ac6ff6693527876387e"
server_key_version: "encryptionkey@0"
I20260924 21:16:08.920011 25384 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:16:08.920686 25399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:08.920852 25384 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:16:08.920950 25384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-2/wal
uuid: "870409d09d654995b8b05d099f9fc7dc"
format_stamp: "Formatted at 2026-09-24 21:16:08 on dist-test-slave-q276"
server_key: "7d34c1979f78d7fbe974747801b8b572"
server_key_iv: "cb558f5b06711ac6ff6693527876387e"
server_key_version: "encryptionkey@0"
I20260924 21:16:08.921032 25384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:08.932542 25384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:08.934444 25384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:08.934612 25384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:08.934854 25384 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:16:08.935166 25384 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:16:08.935220 25384 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:08.935263 25384 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:16:08.935288 25384 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:08.941100 25384 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.131:35987
I20260924 21:16:08.941146 25512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.131:35987 every 8 connection(s)
I20260924 21:16:08.941462 25384 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
I20260924 21:16:08.946100 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 25384
I20260924 21:16:08.946224 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance
I20260924 21:16:08.946468 6986 external_mini_cluster.cc:1428] Setting key 571eebbdb552fdd1c35e5e522b929f58
Sep 24 21:16:08 dist-test-slave-q276 krb5kdc[25009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284568, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:16:08.950989 25093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:08.942165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:60617 (local address 127.6.210.190:33135)
0924 21:16:08.942238 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:08.942240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:08.942459 (+ 219us) server_negotiation.cc:407] Connection header received
0924 21:16:08.942946 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:08.942947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:08.942957 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:08.943017 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:08.943987 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:08.944484 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:08.945465 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:08.945685 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:08.945756 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:08.948396 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:08.948398 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:08.948398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:08.948411 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:08.949677 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:08.950054 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:08.950055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:08.950056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:08.950114 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:08.950382 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:08.950383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:08.950383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:08.950485 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:08.950541 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:08.950860 (+ 319us) server_negotiation.cc:299] Negotiation successful
0924 21:16:08.950926 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260924 21:16:08.951149 25515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:08.942140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:33135 (local address 127.6.210.131:60617)
0924 21:16:08.942328 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:08.942368 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:08.942858 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:08.943144 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:08.943148 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:08.943712 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:08.943902 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:08.943906 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:08.944566 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:08.944567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:08.945354 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:08.945356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:08.945777 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:08.945778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:08.945881 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:08.946567 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:08.946577 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:08.948312 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:08.949748 (+ 1436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:08.949754 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:08.949756 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:08.949969 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:08.950181 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:08.950181 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:08.950181 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:08.950337 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:08.950599 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:08.950605 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:08.950724 (+ 119us) client_negotiation.cc:769] Sending connection context
0924 21:16:08.950813 (+ 89us) client_negotiation.cc:240] Negotiation successful
0924 21:16:08.950937 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":101,"threads_started":1}
I20260924 21:16:08.951653 25513 heartbeater.cc:347] Connected to a master server at 127.6.210.190:33135
I20260924 21:16:08.951756 25513 heartbeater.cc:464] Registering TS with master...
I20260924 21:16:08.951946 25513 heartbeater.cc:511] Master 127.6.210.190:33135 requested a full tablet report, sending...
I20260924 21:16:08.952307 25054 ts_manager.cc:194] Registered new tserver with Master: 870409d09d654995b8b05d099f9fc7dc (127.6.210.131:35987)
I20260924 21:16:08.952682 25054 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.131@KRBTEST.COM'} at 127.6.210.131:60617
I20260924 21:16:08.958365 6986 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:16:08.965605 25093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:08.959662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56328 (local address 127.6.210.190:33135)
0924 21:16:08.959737 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:08.959739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:08.959855 (+ 116us) server_negotiation.cc:407] Connection header received
0924 21:16:08.959896 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:08.959897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:08.959906 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:08.959959 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:08.960485 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:08.960958 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:08.961625 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:08.961797 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:08.961847 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:08.963198 (+ 1351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:08.963199 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:08.963200 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:08.963205 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:08.964607 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:08.964936 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:08.964937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:08.964937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:08.964985 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:08.965142 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:08.965142 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:08.965142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:08.965252 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:08.965323 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:08.965421 (+ 98us) server_negotiation.cc:299] Negotiation successful
0924 21:16:08.965466 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 21:16:08.967159 25054 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56328:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:16:08.968714 25054 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:16:08.975033 25527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:08.971178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49078 (local address 127.6.210.131:35987)
0924 21:16:08.971365 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:08.971570 (+ 205us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:08.971580 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:16:08.971614 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:08.971616 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:08.971628 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:08.971718 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:08.971892 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:08.972626 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:08.973921 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:08.974622 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:08.974707 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:08.974766 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:08.974910 (+ 144us) server_negotiation.cc:299] Negotiation successful
0924 21:16:08.974948 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":85,"threads_started":1}
I20260924 21:16:08.975021 25524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:08.971009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:35987 (local address 127.0.0.1:49078)
0924 21:16:08.971338 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:08.971377 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:08.971444 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:08.971724 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:08.971727 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:08.971730 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:08.971807 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:08.971809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:08.972733 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:08.972734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:08.973825 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:08.973827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:08.974707 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:08.974708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:08.974783 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:08.974796 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 21:16:08.974894 (+ 98us) client_negotiation.cc:240] Negotiation successful
0924 21:16:08.974948 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":75,"threads_started":1}
I20260924 21:16:08.975473 25528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:08.971211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:33059 (local address 127.0.0.1:49256)
0924 21:16:08.971443 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:08.971454 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:08.971510 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:08.971956 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:08.971957 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:08.971959 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:08.971991 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:08.971993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:08.973271 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:08.973272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:08.974217 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:08.974219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:08.975242 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:08.975243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:08.975300 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:08.975303 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:16:08.975348 (+ 45us) client_negotiation.cc:240] Negotiation successful
0924 21:16:08.975388 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":65,"threads_started":1}
I20260924 21:16:08.975708 25525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:08.971009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:43321 (local address 127.0.0.1:46326)
0924 21:16:08.971342 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:08.971384 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:08.971444 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:08.971790 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:08.971791 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:08.971793 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:08.971850 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:08.971852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:08.973241 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:08.973242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:08.974312 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:08.974315 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:08.975476 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:08.975477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:08.975537 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:08.975542 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 21:16:08.975598 (+ 56us) client_negotiation.cc:240] Negotiation successful
0924 21:16:08.975635 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":4,"thread_start_us":55,"threads_started":1}
I20260924 21:16:08.975775 25529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:08.971322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49256 (local address 127.6.210.129:33059)
0924 21:16:08.971597 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:08.971602 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:08.971612 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:16:08.971653 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:08.971655 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:08.971671 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:08.971877 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:08.972107 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:08.973179 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:08.974333 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:08.975053 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:08.975139 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:08.975206 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:08.975645 (+ 439us) server_negotiation.cc:299] Negotiation successful
0924 21:16:08.975692 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":94,"threads_started":1}
I20260924 21:16:08.975809 25447 tablet_service.cc:1467] Processing CreateTablet for tablet 45a4340a76774ac6ab4061f2f0d73a11 (DEFAULT_TABLE table=test-table [id=dcf60475d70a43f0a74fbf9245f03975]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:08.976130 25526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:08.971134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46326 (local address 127.6.210.130:43321)
0924 21:16:08.971604 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:08.971608 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:08.971640 (+ 32us) server_negotiation.cc:407] Connection header received
0924 21:16:08.971675 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:08.971677 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:08.971695 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:08.971928 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:08.972003 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:08.973126 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:08.974391 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:08.975053 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:08.975457 (+ 404us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:08.975694 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:08.975719 (+ 25us) server_negotiation.cc:299] Negotiation successful
0924 21:16:08.976038 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":102,"threads_started":1}
I20260924 21:16:08.976258 25447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45a4340a76774ac6ab4061f2f0d73a11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:08.976766 25313 tablet_service.cc:1467] Processing CreateTablet for tablet 45a4340a76774ac6ab4061f2f0d73a11 (DEFAULT_TABLE table=test-table [id=dcf60475d70a43f0a74fbf9245f03975]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:08.977073 25313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45a4340a76774ac6ab4061f2f0d73a11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:08.978843 25179 tablet_service.cc:1467] Processing CreateTablet for tablet 45a4340a76774ac6ab4061f2f0d73a11 (DEFAULT_TABLE table=test-table [id=dcf60475d70a43f0a74fbf9245f03975]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:08.979203 25179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45a4340a76774ac6ab4061f2f0d73a11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:08.979689 25530 tablet_bootstrap.cc:492] T 45a4340a76774ac6ab4061f2f0d73a11 P 870409d09d654995b8b05d099f9fc7dc: Bootstrap starting.
I20260924 21:16:08.980352 25530 tablet_bootstrap.cc:654] T 45a4340a76774ac6ab4061f2f0d73a11 P 870409d09d654995b8b05d099f9fc7dc: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:08.980433 25531 tablet_bootstrap.cc:492] T 45a4340a76774ac6ab4061f2f0d73a11 P 377952d106c743958279dfe30271e4b0: Bootstrap starting.
I20260924 21:16:08.980937 25531 tablet_bootstrap.cc:654] T 45a4340a76774ac6ab4061f2f0d73a11 P 377952d106c743958279dfe30271e4b0: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:08.980912 25530 log.cc:824] T 45a4340a76774ac6ab4061f2f0d73a11 P 870409d09d654995b8b05d099f9fc7dc: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:08.981446 25531 log.cc:824] T 45a4340a76774ac6ab4061f2f0d73a11 P 377952d106c743958279dfe30271e4b0: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:08.981738 25530 tablet_bootstrap.cc:492] T 45a4340a76774ac6ab4061f2f0d73a11 P 870409d09d654995b8b05d099f9fc7dc: No bootstrap required, opened a new log
I20260924 21:16:08.981797 25530 ts_tablet_manager.cc:1397] T 45a4340a76774ac6ab4061f2f0d73a11 P 870409d09d654995b8b05d099f9fc7dc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:16:08.982071 25531 tablet_bootstrap.cc:492] T 45a4340a76774ac6ab4061f2f0d73a11 P 377952d106c743958279dfe30271e4b0: No bootstrap required, opened a new log
I20260924 21:16:08.982115 25531 ts_tablet_manager.cc:1397] T 45a4340a76774ac6ab4061f2f0d73a11 P 377952d106c743958279dfe30271e4b0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:16:08.983608 25531 raft_consensus.cc:348] T 45a4340a76774ac6ab4061f2f0d73a11 P 377952d106c743958279dfe30271e4b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "377952d106c743958279dfe30271e4b0" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 43321 } } peers { permanent_uuid: "36c458ef6166447380fbce070938173a" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 33059 } } peers { permanent_uuid: "870409d09d654995b8b05d099f9fc7dc" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35987 } }
I20260924 21:16:08.983598 25530 raft_consensus.cc:348] T 45a4340a76774ac6ab4061f2f0d73a11 P 870409d09d654995b8b05d099f9fc7dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "377952d106c743958279dfe30271e4b0" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 43321 } } peers { permanent_uuid: "36c458ef6166447380fbce070938173a" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 33059 } } peers { permanent_uuid: "870409d09d654995b8b05d099f9fc7dc" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35987 } }
I20260924 21:16:08.983744 25531 raft_consensus.cc:374] T 45a4340a76774ac6ab4061f2f0d73a11 P 377952d106c743958279dfe30271e4b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:08.983767 25531 raft_consensus.cc:729] T 45a4340a76774ac6ab4061f2f0d73a11 P 377952d106c743958279dfe30271e4b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 377952d106c743958279dfe30271e4b0, State: Initialized, Role: FOLLOWER
I20260924 21:16:08.983769 25530 raft_consensus.cc:374] T 45a4340a76774ac6ab4061f2f0d73a11 P 870409d09d654995b8b05d099f9fc7dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:08.983808 25530 raft_consensus.cc:729] T 45a4340a76774ac6ab4061f2f0d73a11 P 870409d09d654995b8b05d099f9fc7dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 870409d09d654995b8b05d099f9fc7dc, State: Initialized, Role: FOLLOWER
I20260924 21:16:08.983891 25531 consensus_queue.cc:260] T 45a4340a76774ac6ab4061f2f0d73a11 P 377952d106c743958279dfe30271e4b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "377952d106c743958279dfe30271e4b0" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 43321 } } peers { permanent_uuid: "36c458ef6166447380fbce070938173a" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 33059 } } peers { permanent_uuid: "870409d09d654995b8b05d099f9fc7dc" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35987 } }
I20260924 21:16:08.983978 25530 consensus_queue.cc:260] T 45a4340a76774ac6ab4061f2f0d73a11 P 870409d09d654995b8b05d099f9fc7dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "377952d106c743958279dfe30271e4b0" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 43321 } } peers { permanent_uuid: "36c458ef6166447380fbce070938173a" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 33059 } } peers { permanent_uuid: "870409d09d654995b8b05d099f9fc7dc" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35987 } }
I20260924 21:16:08.984184 25531 ts_tablet_manager.cc:1428] T 45a4340a76774ac6ab4061f2f0d73a11 P 377952d106c743958279dfe30271e4b0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:16:08.984200 25379 heartbeater.cc:503] Master 127.6.210.190:33135 was elected leader, sending a full tablet report...
I20260924 21:16:08.984390 25534 tablet_bootstrap.cc:492] T 45a4340a76774ac6ab4061f2f0d73a11 P 36c458ef6166447380fbce070938173a: Bootstrap starting.
I20260924 21:16:08.984578 25530 ts_tablet_manager.cc:1428] T 45a4340a76774ac6ab4061f2f0d73a11 P 870409d09d654995b8b05d099f9fc7dc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:16:08.984599 25513 heartbeater.cc:503] Master 127.6.210.190:33135 was elected leader, sending a full tablet report...
I20260924 21:16:08.985075 25534 tablet_bootstrap.cc:654] T 45a4340a76774ac6ab4061f2f0d73a11 P 36c458ef6166447380fbce070938173a: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:08.985620 25534 log.cc:824] T 45a4340a76774ac6ab4061f2f0d73a11 P 36c458ef6166447380fbce070938173a: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:08.986223 25534 tablet_bootstrap.cc:492] T 45a4340a76774ac6ab4061f2f0d73a11 P 36c458ef6166447380fbce070938173a: No bootstrap required, opened a new log
I20260924 21:16:08.986330 25534 ts_tablet_manager.cc:1397] T 45a4340a76774ac6ab4061f2f0d73a11 P 36c458ef6166447380fbce070938173a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:16:08.987816 25534 raft_consensus.cc:348] T 45a4340a76774ac6ab4061f2f0d73a11 P 36c458ef6166447380fbce070938173a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "377952d106c743958279dfe30271e4b0" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 43321 } } peers { permanent_uuid: "36c458ef6166447380fbce070938173a" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 33059 } } peers { permanent_uuid: "870409d09d654995b8b05d099f9fc7dc" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35987 } }
I20260924 21:16:08.987949 25534 raft_consensus.cc:374] T 45a4340a76774ac6ab4061f2f0d73a11 P 36c458ef6166447380fbce070938173a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:08.987972 25534 raft_consensus.cc:729] T 45a4340a76774ac6ab4061f2f0d73a11 P 36c458ef6166447380fbce070938173a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36c458ef6166447380fbce070938173a, State: Initialized, Role: FOLLOWER
I20260924 21:16:08.988086 25534 consensus_queue.cc:260] T 45a4340a76774ac6ab4061f2f0d73a11 P 36c458ef6166447380fbce070938173a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "377952d106c743958279dfe30271e4b0" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 43321 } } peers { permanent_uuid: "36c458ef6166447380fbce070938173a" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 33059 } } peers { permanent_uuid: "870409d09d654995b8b05d099f9fc7dc" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35987 } }
I20260924 21:16:08.988350 25534 ts_tablet_manager.cc:1428] T 45a4340a76774ac6ab4061f2f0d73a11 P 36c458ef6166447380fbce070938173a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:16:08.988379 25245 heartbeater.cc:503] Master 127.6.210.190:33135 was elected leader, sending a full tablet report...
W20260924 21:16:09.038492 25380 tablet.cc:2369] T 45a4340a76774ac6ab4061f2f0d73a11 P 377952d106c743958279dfe30271e4b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:16:09.079022 25535 raft_consensus.cc:482] T 45a4340a76774ac6ab4061f2f0d73a11 P 377952d106c743958279dfe30271e4b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:16:09.079159 25535 raft_consensus.cc:504] T 45a4340a76774ac6ab4061f2f0d73a11 P 377952d106c743958279dfe30271e4b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "377952d106c743958279dfe30271e4b0" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 43321 } } peers { permanent_uuid: "36c458ef6166447380fbce070938173a" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 33059 } } peers { permanent_uuid: "870409d09d654995b8b05d099f9fc7dc" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35987 } }
I20260924 21:16:09.079545 25535 leader_election.cc:288] T 45a4340a76774ac6ab4061f2f0d73a11 P 377952d106c743958279dfe30271e4b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 36c458ef6166447380fbce070938173a (127.6.210.129:33059), 870409d09d654995b8b05d099f9fc7dc (127.6.210.131:35987)
I20260924 21:16:09.082932 25527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:09.079869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:41857 (local address 127.6.210.131:35987)
0924 21:16:09.079918 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:09.079920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:09.079950 (+ 30us) server_negotiation.cc:407] Connection header received
0924 21:16:09.079987 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:09.079988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:09.079999 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:09.080070 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:09.080370 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.080954 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.082106 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.082606 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.082666 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:09.082674 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:09.082848 (+ 174us) server_negotiation.cc:299] Negotiation successful
0924 21:16:09.082876 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 21:16:09.082967 25381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:09.079758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:35987 (local address 127.6.210.130:41857)
0924 21:16:09.079833 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:09.079847 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:09.079897 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:09.080147 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:09.080148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:09.080150 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:09.080262 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:09.080264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.081054 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.081055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:09.082000 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:09.082003 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.082687 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.082688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:09.082766 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:09.082770 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:16:09.082832 (+ 62us) client_negotiation.cc:240] Negotiation successful
0924 21:16:09.082873 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260924 21:16:09.083369 25539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:09.079764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:33059 (local address 127.6.210.130:44191)
0924 21:16:09.080038 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:09.080051 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:09.080179 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:09.080367 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:09.080368 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:09.080370 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:09.080442 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:09.080444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.081274 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.081275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:09.082172 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:09.082175 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.082999 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.083000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:09.083148 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:09.083152 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:16:09.083246 (+ 94us) client_negotiation.cc:240] Negotiation successful
0924 21:16:09.083286 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":69,"thread_start_us":110,"threads_started":1}
I20260924 21:16:09.083379 25529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:09.079860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:44191 (local address 127.6.210.129:33059)
0924 21:16:09.079916 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:09.079918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:09.080217 (+ 299us) server_negotiation.cc:407] Connection header received
0924 21:16:09.080254 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:09.080255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:09.080266 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:09.080346 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:09.080554 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.081180 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.082421 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.082912 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.082979 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:09.082987 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:09.083266 (+ 279us) server_negotiation.cc:299] Negotiation successful
0924 21:16:09.083313 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 21:16:09.083364 25467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45a4340a76774ac6ab4061f2f0d73a11" candidate_uuid: "377952d106c743958279dfe30271e4b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "870409d09d654995b8b05d099f9fc7dc" is_pre_election: true
I20260924 21:16:09.083537 25467 raft_consensus.cc:2454] T 45a4340a76774ac6ab4061f2f0d73a11 P 870409d09d654995b8b05d099f9fc7dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 377952d106c743958279dfe30271e4b0 in term 0.
I20260924 21:16:09.083783 25199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45a4340a76774ac6ab4061f2f0d73a11" candidate_uuid: "377952d106c743958279dfe30271e4b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36c458ef6166447380fbce070938173a" is_pre_election: true
I20260924 21:16:09.083834 25268 leader_election.cc:302] T 45a4340a76774ac6ab4061f2f0d73a11 P 377952d106c743958279dfe30271e4b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 377952d106c743958279dfe30271e4b0, 870409d09d654995b8b05d099f9fc7dc; no voters:
I20260924 21:16:09.083937 25199 raft_consensus.cc:2454] T 45a4340a76774ac6ab4061f2f0d73a11 P 36c458ef6166447380fbce070938173a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 377952d106c743958279dfe30271e4b0 in term 0.
I20260924 21:16:09.083976 25535 raft_consensus.cc:2781] T 45a4340a76774ac6ab4061f2f0d73a11 P 377952d106c743958279dfe30271e4b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:16:09.084054 25535 raft_consensus.cc:482] T 45a4340a76774ac6ab4061f2f0d73a11 P 377952d106c743958279dfe30271e4b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:16:09.084096 25535 raft_consensus.cc:3037] T 45a4340a76774ac6ab4061f2f0d73a11 P 377952d106c743958279dfe30271e4b0 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:09.084849 25535 raft_consensus.cc:504] T 45a4340a76774ac6ab4061f2f0d73a11 P 377952d106c743958279dfe30271e4b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "377952d106c743958279dfe30271e4b0" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 43321 } } peers { permanent_uuid: "36c458ef6166447380fbce070938173a" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 33059 } } peers { permanent_uuid: "870409d09d654995b8b05d099f9fc7dc" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35987 } }
I20260924 21:16:09.085001 25535 leader_election.cc:288] T 45a4340a76774ac6ab4061f2f0d73a11 P 377952d106c743958279dfe30271e4b0 [CANDIDATE]: Term 1 election: Requested vote from peers 36c458ef6166447380fbce070938173a (127.6.210.129:33059), 870409d09d654995b8b05d099f9fc7dc (127.6.210.131:35987)
I20260924 21:16:09.085211 25467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45a4340a76774ac6ab4061f2f0d73a11" candidate_uuid: "377952d106c743958279dfe30271e4b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "870409d09d654995b8b05d099f9fc7dc"
I20260924 21:16:09.085251 25199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45a4340a76774ac6ab4061f2f0d73a11" candidate_uuid: "377952d106c743958279dfe30271e4b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36c458ef6166447380fbce070938173a"
I20260924 21:16:09.085330 25467 raft_consensus.cc:3037] T 45a4340a76774ac6ab4061f2f0d73a11 P 870409d09d654995b8b05d099f9fc7dc [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:09.085327 25199 raft_consensus.cc:3037] T 45a4340a76774ac6ab4061f2f0d73a11 P 36c458ef6166447380fbce070938173a [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:09.086242 25467 raft_consensus.cc:2454] T 45a4340a76774ac6ab4061f2f0d73a11 P 870409d09d654995b8b05d099f9fc7dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 377952d106c743958279dfe30271e4b0 in term 1.
I20260924 21:16:09.086380 25199 raft_consensus.cc:2454] T 45a4340a76774ac6ab4061f2f0d73a11 P 36c458ef6166447380fbce070938173a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 377952d106c743958279dfe30271e4b0 in term 1.
I20260924 21:16:09.086426 25268 leader_election.cc:302] T 45a4340a76774ac6ab4061f2f0d73a11 P 377952d106c743958279dfe30271e4b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 377952d106c743958279dfe30271e4b0, 870409d09d654995b8b05d099f9fc7dc; no voters:
I20260924 21:16:09.086519 25535 raft_consensus.cc:2781] T 45a4340a76774ac6ab4061f2f0d73a11 P 377952d106c743958279dfe30271e4b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:09.086711 25535 raft_consensus.cc:686] T 45a4340a76774ac6ab4061f2f0d73a11 P 377952d106c743958279dfe30271e4b0 [term 1 LEADER]: Becoming Leader. State: Replica: 377952d106c743958279dfe30271e4b0, State: Running, Role: LEADER
I20260924 21:16:09.086830 25535 consensus_queue.cc:237] T 45a4340a76774ac6ab4061f2f0d73a11 P 377952d106c743958279dfe30271e4b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "377952d106c743958279dfe30271e4b0" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 43321 } } peers { permanent_uuid: "36c458ef6166447380fbce070938173a" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 33059 } } peers { permanent_uuid: "870409d09d654995b8b05d099f9fc7dc" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35987 } }
I20260924 21:16:09.087716 25054 catalog_manager.cc:5495] T 45a4340a76774ac6ab4061f2f0d73a11 P 377952d106c743958279dfe30271e4b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 377952d106c743958279dfe30271e4b0 (127.6.210.130). New cstate: current_term: 1 leader_uuid: "377952d106c743958279dfe30271e4b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "377952d106c743958279dfe30271e4b0" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 43321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36c458ef6166447380fbce070938173a" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 33059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "870409d09d654995b8b05d099f9fc7dc" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35987 } health_report { overall_health: UNKNOWN } } }
I20260924 21:16:09.120198 25526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:09.117415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46334 (local address 127.6.210.130:43321)
0924 21:16:09.117474 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:09.117476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:09.117486 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:16:09.117602 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:09.117603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:09.117637 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:09.117726 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:16:09.117868 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.118480 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.119327 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.119570 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.119655 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:09.119803 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:16:09.119908 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:16:09.120031 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:09.120065 (+ 34us) server_negotiation.cc:299] Negotiation successful
0924 21:16:09.120078 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 21:16:09.122098 25467 raft_consensus.cc:1261] T 45a4340a76774ac6ab4061f2f0d73a11 P 870409d09d654995b8b05d099f9fc7dc [term 1 FOLLOWER]: Refusing update from remote peer 377952d106c743958279dfe30271e4b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:16:09.122135 25199 raft_consensus.cc:1261] T 45a4340a76774ac6ab4061f2f0d73a11 P 36c458ef6166447380fbce070938173a [term 1 FOLLOWER]: Refusing update from remote peer 377952d106c743958279dfe30271e4b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:16:09.122510 25540 consensus_queue.cc:1035] T 45a4340a76774ac6ab4061f2f0d73a11 P 377952d106c743958279dfe30271e4b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "870409d09d654995b8b05d099f9fc7dc" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 35987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:16:09.122710 25535 consensus_queue.cc:1035] T 45a4340a76774ac6ab4061f2f0d73a11 P 377952d106c743958279dfe30271e4b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36c458ef6166447380fbce070938173a" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 33059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:16:09.125033 25543 mvcc.cc:205] Tried to move back new op lower bound from 7333005595121274880 to 7333005594980372480. Current Snapshot: MvccSnapshot[applied={T|T < 7333005595121274880}]
I20260924 21:16:09.125728 25545 mvcc.cc:205] Tried to move back new op lower bound from 7333005595121274880 to 7333005594980372480. Current Snapshot: MvccSnapshot[applied={T|T < 7333005595121274880}]
I20260924 21:16:09.126315 25544 mvcc.cc:205] Tried to move back new op lower bound from 7333005595121274880 to 7333005594980372480. Current Snapshot: MvccSnapshot[applied={T|T < 7333005595121274880 or (T in {7333005595121274880})}]
I20260924 21:16:09.127677 25053 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:16:09.127777 25053 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:16:09.128566 25053 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 41e8e56b81d0467f8fc036b353c3e3a2: Sending DeleteTablet for 3 replicas of tablet 45a4340a76774ac6ab4061f2f0d73a11
I20260924 21:16:09.128908 25313 tablet_service.cc:1514] Processing DeleteTablet for tablet 45a4340a76774ac6ab4061f2f0d73a11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:09 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:46326
I20260924 21:16:09.128980 25179 tablet_service.cc:1514] Processing DeleteTablet for tablet 45a4340a76774ac6ab4061f2f0d73a11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:09 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:49256
I20260924 21:16:09.128997 25447 tablet_service.cc:1514] Processing DeleteTablet for tablet 45a4340a76774ac6ab4061f2f0d73a11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:09 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:49078
I20260924 21:16:09.129217 25553 tablet_replica.cc:330] T 45a4340a76774ac6ab4061f2f0d73a11 P 36c458ef6166447380fbce070938173a: stopping tablet replica
I20260924 21:16:09.129321 25553 raft_consensus.cc:2229] T 45a4340a76774ac6ab4061f2f0d73a11 P 36c458ef6166447380fbce070938173a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:16:09.129431 25553 raft_consensus.cc:2258] T 45a4340a76774ac6ab4061f2f0d73a11 P 36c458ef6166447380fbce070938173a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:16:09.129462 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 25116
I20260924 21:16:09.130062 25554 tablet_replica.cc:330] T 45a4340a76774ac6ab4061f2f0d73a11 P 870409d09d654995b8b05d099f9fc7dc: stopping tablet replica
I20260924 21:16:09.130170 25554 raft_consensus.cc:2229] T 45a4340a76774ac6ab4061f2f0d73a11 P 870409d09d654995b8b05d099f9fc7dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:16:09.130298 25554 raft_consensus.cc:2258] T 45a4340a76774ac6ab4061f2f0d73a11 P 870409d09d654995b8b05d099f9fc7dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:16:09.130780 25554 ts_tablet_manager.cc:1905] T 45a4340a76774ac6ab4061f2f0d73a11 P 870409d09d654995b8b05d099f9fc7dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:16:09.132939 25554 ts_tablet_manager.cc:1918] T 45a4340a76774ac6ab4061f2f0d73a11 P 870409d09d654995b8b05d099f9fc7dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:16:09.133101 25554 log.cc:1196] T 45a4340a76774ac6ab4061f2f0d73a11 P 870409d09d654995b8b05d099f9fc7dc: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284333163253-6986-0/minicluster-data/ts-2/wal/wals/45a4340a76774ac6ab4061f2f0d73a11
I20260924 21:16:09.133491 25554 ts_tablet_manager.cc:1939] T 45a4340a76774ac6ab4061f2f0d73a11 P 870409d09d654995b8b05d099f9fc7dc: Deleting consensus metadata
I20260924 21:16:09.133988 25040 catalog_manager.cc:4841] TS 870409d09d654995b8b05d099f9fc7dc (127.6.210.131:35987): tablet 45a4340a76774ac6ab4061f2f0d73a11 (table test-table [id=dcf60475d70a43f0a74fbf9245f03975]) successfully deleted
I20260924 21:16:09.138422 25552 tablet_replica.cc:330] T 45a4340a76774ac6ab4061f2f0d73a11 P 377952d106c743958279dfe30271e4b0: stopping tablet replica
I20260924 21:16:09.138579 25552 raft_consensus.cc:2229] T 45a4340a76774ac6ab4061f2f0d73a11 P 377952d106c743958279dfe30271e4b0 [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:16:09.138715 25552 raft_consensus.cc:2258] T 45a4340a76774ac6ab4061f2f0d73a11 P 377952d106c743958279dfe30271e4b0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 21:16:09.138893 25039 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.210.129:33059 (error 108)
I20260924 21:16:09.139115 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 25250
I20260924 21:16:09.139237 25552 ts_tablet_manager.cc:1905] T 45a4340a76774ac6ab4061f2f0d73a11 P 377952d106c743958279dfe30271e4b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:16:09.143306 25525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:09.143113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:33059 (local address 127.0.0.1:49268)
0924 21:16:09.143203 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:09.143256 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.129:33059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260924 21:16:09.143477 25039 catalog_manager.cc:4584] TS 36c458ef6166447380fbce070938173a (127.6.210.129:33059): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 45a4340a76774ac6ab4061f2f0d73a11: Network error: Client connection negotiation failed: client connection to 127.6.210.129:33059: connect: Connection refused (error 111)
I20260924 21:16:09.144549 25525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:09.144420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:43321 (local address 127.0.0.1:46348)
0924 21:16:09.144489 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:09.144520 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.130:43321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260924 21:16:09.144547 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 25384
I20260924 21:16:09.148681 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 25022
2026-09-24T21:16:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2013 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-24T21:16:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:16:09Z Disabled control of system clock
I20260924 21:16:09.177325 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:37681
--webserver_interface=127.6.210.190
--webserver_port=0
--builtin_ntp_servers=127.6.210.148:37331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:37681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 21:16:09.256592 25559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:09.256808 25559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:09.256829 25559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:09.258004 25559 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:09.258049 25559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:09.258064 25559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:09.258075 25559 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:09.258088 25559 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:09.259498 25559 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:37331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:37681
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:37681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:09.259755 25559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:09.260059 25559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:09.265046 25567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:09.265034 25565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:09.265259 25559 server_base.cc:1034] running on GCE node
W20260924 21:16:09.264967 25564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:09.265584 25559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:09.265830 25559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:09.266996 25559 hybrid_clock.cc:648] HybridClock initialized: now 1790284569266979 us; error 47 us; skew 500 ppm
I20260924 21:16:09.268504 25559 webserver.cc:460] Webserver started at http://127.6.210.190:45495/ using document root <none> and password file <none>
I20260924 21:16:09.268772 25559 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:09.268819 25559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:09.268910 25559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:09.269961 25559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "589a3dbb24ae4cdc91fa48c1814bcc6f"
format_stamp: "Formatted at 2026-09-24 21:16:09 on dist-test-slave-q276"
server_key: "00ef89d3c23eab15fc91a5563de09f79"
server_key_iv: "8f52efb361ff5624c99cc0d3a654ed54"
server_key_version: "encryptionkey@0"
I20260924 21:16:09.270334 25559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "589a3dbb24ae4cdc91fa48c1814bcc6f"
format_stamp: "Formatted at 2026-09-24 21:16:09 on dist-test-slave-q276"
server_key: "00ef89d3c23eab15fc91a5563de09f79"
server_key_iv: "8f52efb361ff5624c99cc0d3a654ed54"
server_key_version: "encryptionkey@0"
I20260924 21:16:09.271565 25559 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:16:09.272254 25573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:09.272423 25559 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:16:09.272511 25559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "589a3dbb24ae4cdc91fa48c1814bcc6f"
format_stamp: "Formatted at 2026-09-24 21:16:09 on dist-test-slave-q276"
server_key: "00ef89d3c23eab15fc91a5563de09f79"
server_key_iv: "8f52efb361ff5624c99cc0d3a654ed54"
server_key_version: "encryptionkey@0"
I20260924 21:16:09.272593 25559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:09.284952 25559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:09.285271 25559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:09.285426 25559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:09.289542 25559 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:37681
I20260924 21:16:09.289578 25625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:37681 every 8 connection(s)
I20260924 21:16:09.289944 25559 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:16:09.290933 25626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:09.291740 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 25559
I20260924 21:16:09.291849 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:16:09.292065 6986 external_mini_cluster.cc:1428] Setting key 2ac5a3f9e814813fd6bb8f7c17cab553
I20260924 21:16:09.294320 25626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 589a3dbb24ae4cdc91fa48c1814bcc6f: Bootstrap starting.
I20260924 21:16:09.295012 25626 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 589a3dbb24ae4cdc91fa48c1814bcc6f: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:09.295310 25626 log.cc:824] T 00000000000000000000000000000000 P 589a3dbb24ae4cdc91fa48c1814bcc6f: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:09.296041 25626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 589a3dbb24ae4cdc91fa48c1814bcc6f: No bootstrap required, opened a new log
I20260924 21:16:09.298135 25626 raft_consensus.cc:348] T 00000000000000000000000000000000 P 589a3dbb24ae4cdc91fa48c1814bcc6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589a3dbb24ae4cdc91fa48c1814bcc6f" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37681 } }
I20260924 21:16:09.298321 25629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:09.292643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55624 (local address 127.6.210.190:37681)
0924 21:16:09.292869 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:09.292879 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:09.292911 (+ 32us) server_negotiation.cc:407] Connection header received
0924 21:16:09.293441 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:09.293450 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:09.293532 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:09.293686 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:09.293841 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.295198 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.295970 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.296564 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.296687 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:09.297330 (+ 643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:09.297334 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:09.297334 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:09.297535 (+ 201us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:09.297577 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:09.297591 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:09.297660 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:09.297731 (+ 71us) server_negotiation.cc:299] Negotiation successful
0924 21:16:09.297811 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":90,"threads_started":1}
I20260924 21:16:09.298354 25626 raft_consensus.cc:374] T 00000000000000000000000000000000 P 589a3dbb24ae4cdc91fa48c1814bcc6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:09.298393 25626 raft_consensus.cc:729] T 00000000000000000000000000000000 P 589a3dbb24ae4cdc91fa48c1814bcc6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 589a3dbb24ae4cdc91fa48c1814bcc6f, State: Initialized, Role: FOLLOWER
I20260924 21:16:09.298542 25626 consensus_queue.cc:260] T 00000000000000000000000000000000 P 589a3dbb24ae4cdc91fa48c1814bcc6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589a3dbb24ae4cdc91fa48c1814bcc6f" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37681 } }
I20260924 21:16:09.298641 25626 raft_consensus.cc:388] T 00000000000000000000000000000000 P 589a3dbb24ae4cdc91fa48c1814bcc6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:16:09.298686 25626 raft_consensus.cc:482] T 00000000000000000000000000000000 P 589a3dbb24ae4cdc91fa48c1814bcc6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:16:09.298779 25626 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 589a3dbb24ae4cdc91fa48c1814bcc6f [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:09.299394 25626 raft_consensus.cc:504] T 00000000000000000000000000000000 P 589a3dbb24ae4cdc91fa48c1814bcc6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589a3dbb24ae4cdc91fa48c1814bcc6f" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37681 } }
I20260924 21:16:09.299484 25626 leader_election.cc:302] T 00000000000000000000000000000000 P 589a3dbb24ae4cdc91fa48c1814bcc6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 589a3dbb24ae4cdc91fa48c1814bcc6f; no voters:
I20260924 21:16:09.299609 25626 leader_election.cc:288] T 00000000000000000000000000000000 P 589a3dbb24ae4cdc91fa48c1814bcc6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:16:09.299679 25631 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 589a3dbb24ae4cdc91fa48c1814bcc6f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:09.299880 25626 sys_catalog.cc:564] T 00000000000000000000000000000000 P 589a3dbb24ae4cdc91fa48c1814bcc6f [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:09.299893 25631 raft_consensus.cc:686] T 00000000000000000000000000000000 P 589a3dbb24ae4cdc91fa48c1814bcc6f [term 1 LEADER]: Becoming Leader. State: Replica: 589a3dbb24ae4cdc91fa48c1814bcc6f, State: Running, Role: LEADER
I20260924 21:16:09.300199 25631 consensus_queue.cc:237] T 00000000000000000000000000000000 P 589a3dbb24ae4cdc91fa48c1814bcc6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589a3dbb24ae4cdc91fa48c1814bcc6f" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37681 } }
I20260924 21:16:09.300623 25632 sys_catalog.cc:455] T 00000000000000000000000000000000 P 589a3dbb24ae4cdc91fa48c1814bcc6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "589a3dbb24ae4cdc91fa48c1814bcc6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589a3dbb24ae4cdc91fa48c1814bcc6f" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37681 } } }
I20260924 21:16:09.300717 25632 sys_catalog.cc:458] T 00000000000000000000000000000000 P 589a3dbb24ae4cdc91fa48c1814bcc6f [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:09.300925 25640 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:16:09.300936 25633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 589a3dbb24ae4cdc91fa48c1814bcc6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 589a3dbb24ae4cdc91fa48c1814bcc6f. Latest consensus state: current_term: 1 leader_uuid: "589a3dbb24ae4cdc91fa48c1814bcc6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "589a3dbb24ae4cdc91fa48c1814bcc6f" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37681 } } }
I20260924 21:16:09.301173 25633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 589a3dbb24ae4cdc91fa48c1814bcc6f [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:09.301606 25640 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:16:09.303162 25640 catalog_manager.cc:1347] Generated new cluster ID: 315dd6b9ef38464996c2019bddc78702
I20260924 21:16:09.303236 25640 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:16:09.311393 25640 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:16:09.311967 25640 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:16:09.315584 25640 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 589a3dbb24ae4cdc91fa48c1814bcc6f: Generated new TSK 0
I20260924 21:16:09.315768 25640 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:16:09.320703 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.129:0
--local_ip_for_outbound_sockets=127.6.210.129
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:37681
--builtin_ntp_servers=127.6.210.148:37331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 21:16:09.411799 25650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:09.412012 25650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:09.412035 25650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:09.413195 25650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:09.413235 25650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:09.413261 25650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.129
I20260924 21:16:09.414639 25650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:37331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:37681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.129
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:09.414903 25650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:09.415211 25650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:09.421227 25656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:09.421238 25655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:09.421237 25658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:09.421988 25650 server_base.cc:1034] running on GCE node
I20260924 21:16:09.422252 25650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:09.422562 25650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:09.423702 25650 hybrid_clock.cc:648] HybridClock initialized: now 1790284569423689 us; error 30 us; skew 500 ppm
I20260924 21:16:09.424965 25650 webserver.cc:460] Webserver started at http://127.6.210.129:46521/ using document root <none> and password file <none>
I20260924 21:16:09.425208 25650 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:09.425254 25650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:09.425382 25650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:09.426644 25650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-0/data/instance:
uuid: "335fffa14632475aacf6724758730fad"
format_stamp: "Formatted at 2026-09-24 21:16:09 on dist-test-slave-q276"
server_key: "d1d410c221304bce343280c553d7f48e"
server_key_iv: "a9296ce38d12ea13ea0b7561a65e48d2"
server_key_version: "encryptionkey@0"
I20260924 21:16:09.427003 25650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance:
uuid: "335fffa14632475aacf6724758730fad"
format_stamp: "Formatted at 2026-09-24 21:16:09 on dist-test-slave-q276"
server_key: "d1d410c221304bce343280c553d7f48e"
server_key_iv: "a9296ce38d12ea13ea0b7561a65e48d2"
server_key_version: "encryptionkey@0"
I20260924 21:16:09.428591 25650 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:16:09.429921 25664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:09.430102 25650 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 21:16:09.430177 25650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-0/wal
uuid: "335fffa14632475aacf6724758730fad"
format_stamp: "Formatted at 2026-09-24 21:16:09 on dist-test-slave-q276"
server_key: "d1d410c221304bce343280c553d7f48e"
server_key_iv: "a9296ce38d12ea13ea0b7561a65e48d2"
server_key_version: "encryptionkey@0"
I20260924 21:16:09.430243 25650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:09.447028 25650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:09.447363 25650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:09.447499 25650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:09.447745 25650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:16:09.448109 25650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:16:09.448151 25650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:09.448179 25650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:16:09.448195 25650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:09.454429 25650 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.129:40597
I20260924 21:16:09.454483 25777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.129:40597 every 8 connection(s)
I20260924 21:16:09.454836 25650 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
I20260924 21:16:09.456256 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 25650
I20260924 21:16:09.456364 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance
I20260924 21:16:09.456557 6986 external_mini_cluster.cc:1428] Setting key fbfe3ae80b1a61e41e18aaef79fddea4
I20260924 21:16:09.457520 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.130:0
--local_ip_for_outbound_sockets=127.6.210.130
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:37681
--builtin_ntp_servers=127.6.210.148:37331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260924 21:16:09.460479 25629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:09.455595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:41251 (local address 127.6.210.190:37681)
0924 21:16:09.455662 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:09.455664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:09.455928 (+ 264us) server_negotiation.cc:407] Connection header received
0924 21:16:09.456480 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:09.456481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:09.456495 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:09.456557 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:09.457400 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.457986 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.459070 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.459262 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.459338 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:09.460178 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:09.460179 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:09.460179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:09.460184 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:09.460202 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:09.460206 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:09.460267 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:09.460370 (+ 103us) server_negotiation.cc:299] Negotiation successful
0924 21:16:09.460400 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 21:16:09.460680 25780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:09.455582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37681 (local address 127.6.210.129:41251)
0924 21:16:09.455802 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:09.455837 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:09.456367 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:09.456648 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:09.456651 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:09.456659 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:09.457305 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:09.457317 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.458085 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.458086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:09.458959 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:09.458961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.459346 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.459347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:09.459479 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:09.460037 (+ 558us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:16:09.460037 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:09.460066 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:16:09.460089 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:09.460273 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:09.460278 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:09.460280 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 21:16:09.460356 (+ 76us) client_negotiation.cc:240] Negotiation successful
0924 21:16:09.460457 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":107,"threads_started":1}
I20260924 21:16:09.461174 25778 heartbeater.cc:347] Connected to a master server at 127.6.210.190:37681
I20260924 21:16:09.461279 25778 heartbeater.cc:464] Registering TS with master...
I20260924 21:16:09.461470 25778 heartbeater.cc:511] Master 127.6.210.190:37681 requested a full tablet report, sending...
I20260924 21:16:09.462039 25590 ts_manager.cc:194] Registered new tserver with Master: 335fffa14632475aacf6724758730fad (127.6.210.129:40597)
I20260924 21:16:09.463039 25590 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.210.129:41251
W20260924 21:16:09.538245 25781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:09.538528 25781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:09.538549 25781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:09.539657 25781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:09.539693 25781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:09.539718 25781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.130
I20260924 21:16:09.540958 25781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:37331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:37681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.130
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:09.541141 25781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:09.541369 25781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:09.546262 25787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:09.546293 25786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:09.546368 25789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:09.546569 25781 server_base.cc:1034] running on GCE node
I20260924 21:16:09.546778 25781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:09.547016 25781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:09.548168 25781 hybrid_clock.cc:648] HybridClock initialized: now 1790284569548142 us; error 35 us; skew 500 ppm
I20260924 21:16:09.549427 25781 webserver.cc:460] Webserver started at http://127.6.210.130:41251/ using document root <none> and password file <none>
I20260924 21:16:09.549700 25781 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:09.549751 25781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:09.549870 25781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:09.550988 25781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-1/data/instance:
uuid: "095c821f2a9c44a2b1fb2cfe40632f2f"
format_stamp: "Formatted at 2026-09-24 21:16:09 on dist-test-slave-q276"
server_key: "fc9a9e99ccc668c480e7654c7d9efa2a"
server_key_iv: "17d994aa663a9b703d770a4b2ad18973"
server_key_version: "encryptionkey@0"
I20260924 21:16:09.551343 25781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance:
uuid: "095c821f2a9c44a2b1fb2cfe40632f2f"
format_stamp: "Formatted at 2026-09-24 21:16:09 on dist-test-slave-q276"
server_key: "fc9a9e99ccc668c480e7654c7d9efa2a"
server_key_iv: "17d994aa663a9b703d770a4b2ad18973"
server_key_version: "encryptionkey@0"
I20260924 21:16:09.552592 25781 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 21:16:09.553318 25795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:09.553531 25781 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:16:09.553638 25781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-1/wal
uuid: "095c821f2a9c44a2b1fb2cfe40632f2f"
format_stamp: "Formatted at 2026-09-24 21:16:09 on dist-test-slave-q276"
server_key: "fc9a9e99ccc668c480e7654c7d9efa2a"
server_key_iv: "17d994aa663a9b703d770a4b2ad18973"
server_key_version: "encryptionkey@0"
I20260924 21:16:09.553712 25781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:09.574826 25781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:09.575162 25781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:09.575320 25781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:09.575635 25781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:16:09.575995 25781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:16:09.576049 25781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:09.576092 25781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:16:09.576119 25781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:09.581698 25781 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.130:40167
I20260924 21:16:09.581789 25908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.130:40167 every 8 connection(s)
I20260924 21:16:09.582119 25781 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
I20260924 21:16:09.582715 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 25781
I20260924 21:16:09.582823 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance
I20260924 21:16:09.583066 6986 external_mini_cluster.cc:1428] Setting key d6b0b4b3e6ec42eeaacd4f6657b4d000
I20260924 21:16:09.584153 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.131:0
--local_ip_for_outbound_sockets=127.6.210.131
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:37681
--builtin_ntp_servers=127.6.210.148:37331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260924 21:16:09.587891 25629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:09.582872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:51159 (local address 127.6.210.190:37681)
0924 21:16:09.583090 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:09.583092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:09.583294 (+ 202us) server_negotiation.cc:407] Connection header received
0924 21:16:09.583921 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:09.583922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:09.583932 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:09.583965 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:09.584895 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.585371 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.586446 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.586657 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.586737 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:09.587610 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:09.587611 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:09.587611 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:09.587617 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:09.587645 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:09.587649 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:09.587708 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:09.587782 (+ 74us) server_negotiation.cc:299] Negotiation successful
0924 21:16:09.587812 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194}
I20260924 21:16:09.588114 25911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:09.582891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37681 (local address 127.6.210.130:51159)
0924 21:16:09.583137 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:09.583188 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:09.583804 (+ 616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:09.584075 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:09.584080 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:09.584091 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:09.584805 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:09.584812 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.585476 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.585477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:09.586355 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:09.586358 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.586756 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.586757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:09.586902 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:09.587455 (+ 553us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:16:09.587455 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:09.587484 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:16:09.587507 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:09.587709 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:09.587716 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:09.587719 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:16:09.587787 (+ 68us) client_negotiation.cc:240] Negotiation successful
0924 21:16:09.587888 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":78,"threads_started":1}
I20260924 21:16:09.588660 25909 heartbeater.cc:347] Connected to a master server at 127.6.210.190:37681
I20260924 21:16:09.588739 25909 heartbeater.cc:464] Registering TS with master...
I20260924 21:16:09.588932 25909 heartbeater.cc:511] Master 127.6.210.190:37681 requested a full tablet report, sending...
I20260924 21:16:09.589332 25590 ts_manager.cc:194] Registered new tserver with Master: 095c821f2a9c44a2b1fb2cfe40632f2f (127.6.210.130:40167)
I20260924 21:16:09.589881 25590 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.210.130:51159
W20260924 21:16:09.667390 25912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:09.667618 25912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:09.667642 25912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:09.668927 25912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:09.668967 25912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:09.669019 25912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.131
I20260924 21:16:09.670639 25912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:37331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:37681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.131
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:09.670892 25912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:09.671217 25912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:09.676694 25920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:09.676808 25917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:09.676847 25912 server_base.cc:1034] running on GCE node
W20260924 21:16:09.676690 25918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:09.677213 25912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:09.677460 25912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:09.678620 25912 hybrid_clock.cc:648] HybridClock initialized: now 1790284569678614 us; error 39 us; skew 500 ppm
I20260924 21:16:09.679951 25912 webserver.cc:460] Webserver started at http://127.6.210.131:33693/ using document root <none> and password file <none>
I20260924 21:16:09.680208 25912 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:09.680282 25912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:09.680435 25912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:09.681495 25912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-2/data/instance:
uuid: "dfb96de7e9164b05bf8407d1343063db"
format_stamp: "Formatted at 2026-09-24 21:16:09 on dist-test-slave-q276"
server_key: "c02c5a914d0a7c7182532368b40a54a9"
server_key_iv: "f72605815c4e51a95721f97a0aff5a3f"
server_key_version: "encryptionkey@0"
I20260924 21:16:09.681854 25912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance:
uuid: "dfb96de7e9164b05bf8407d1343063db"
format_stamp: "Formatted at 2026-09-24 21:16:09 on dist-test-slave-q276"
server_key: "c02c5a914d0a7c7182532368b40a54a9"
server_key_iv: "f72605815c4e51a95721f97a0aff5a3f"
server_key_version: "encryptionkey@0"
I20260924 21:16:09.683236 25912 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:16:09.683916 25926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:09.684082 25912 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:16:09.684204 25912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-2/wal
uuid: "dfb96de7e9164b05bf8407d1343063db"
format_stamp: "Formatted at 2026-09-24 21:16:09 on dist-test-slave-q276"
server_key: "c02c5a914d0a7c7182532368b40a54a9"
server_key_iv: "f72605815c4e51a95721f97a0aff5a3f"
server_key_version: "encryptionkey@0"
I20260924 21:16:09.684279 25912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:09.708613 25912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:09.708941 25912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:09.709101 25912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:09.709386 25912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:16:09.709715 25912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:16:09.709770 25912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:09.709815 25912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:16:09.709849 25912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:09.715572 25912 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.131:43063
I20260924 21:16:09.715616 26039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.131:43063 every 8 connection(s)
I20260924 21:16:09.715946 25912 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
I20260924 21:16:09.719249 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 25912
I20260924 21:16:09.719331 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance
I20260924 21:16:09.719516 6986 external_mini_cluster.cc:1428] Setting key ea0670bb6720565ba87909429e207e83
I20260924 21:16:09.721289 25629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:09.716688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:44749 (local address 127.6.210.190:37681)
0924 21:16:09.716757 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:09.716759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:09.717016 (+ 257us) server_negotiation.cc:407] Connection header received
0924 21:16:09.717476 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:09.717477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:09.717487 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:09.717541 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:09.718199 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.718765 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.719862 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.720170 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.720239 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:09.721075 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:09.721076 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:09.721076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:09.721080 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:09.721093 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:09.721097 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:09.721143 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:09.721216 (+ 73us) server_negotiation.cc:299] Negotiation successful
0924 21:16:09.721236 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 21:16:09.721530 26042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:09.716689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:37681 (local address 127.6.210.131:44749)
0924 21:16:09.716890 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:09.716924 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:09.717388 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:09.717632 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:09.717635 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:09.717644 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:09.718125 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:09.718128 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.718839 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.718840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:09.719786 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:09.719788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.720252 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.720253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:09.720404 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:09.720947 (+ 543us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:16:09.720947 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:09.720974 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:16:09.720995 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:09.721152 (+ 157us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:09.721155 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:09.721158 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:16:09.721209 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 21:16:09.721280 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":87,"threads_started":1}
I20260924 21:16:09.722021 26040 heartbeater.cc:347] Connected to a master server at 127.6.210.190:37681
I20260924 21:16:09.722146 26040 heartbeater.cc:464] Registering TS with master...
I20260924 21:16:09.722388 26040 heartbeater.cc:511] Master 127.6.210.190:37681 requested a full tablet report, sending...
I20260924 21:16:09.722764 25590 ts_manager.cc:194] Registered new tserver with Master: dfb96de7e9164b05bf8407d1343063db (127.6.210.131:43063)
I20260924 21:16:09.723076 25590 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.210.131:44749
I20260924 21:16:09.731416 6986 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:16:09.736402 25629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:09.732962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55626 (local address 127.6.210.190:37681)
0924 21:16:09.733028 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:09.733030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:09.733130 (+ 100us) server_negotiation.cc:407] Connection header received
0924 21:16:09.733196 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:09.733197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:09.733206 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:09.733245 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:09.733359 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.733814 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.734804 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.735068 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.735147 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:09.735955 (+ 808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:09.735957 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:09.735957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:09.735961 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:09.735976 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:09.735980 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:09.736129 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:09.736321 (+ 192us) server_negotiation.cc:299] Negotiation successful
0924 21:16:09.736340 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 21:16:09.738214 25590 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55626:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:16:09.739985 25590 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:16:09.746594 26049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:09.742215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:40597 (local address 127.0.0.1:41244)
0924 21:16:09.742544 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:09.742553 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:09.742593 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:09.743054 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:09.743057 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:09.743059 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:09.743144 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:09.743146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.744525 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.744526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:09.745423 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:09.745426 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.746330 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.746331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:09.746400 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:09.746418 (+ 18us) client_negotiation.cc:769] Sending connection context
0924 21:16:09.746484 (+ 66us) client_negotiation.cc:240] Negotiation successful
0924 21:16:09.746534 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":67,"thread_start_us":83,"threads_started":1}
I20260924 21:16:09.746623 26052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:09.742633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41244 (local address 127.6.210.129:40597)
0924 21:16:09.742853 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:09.742858 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:09.742869 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:16:09.742909 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:09.742911 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:09.742926 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:09.743239 (+ 313us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:09.743303 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.744437 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.745524 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.746166 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.746258 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:09.746379 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:09.746502 (+ 123us) server_negotiation.cc:299] Negotiation successful
0924 21:16:09.746540 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":117,"threads_started":1}
I20260924 21:16:09.747172 26048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:09.742215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:43063 (local address 127.0.0.1:57692)
0924 21:16:09.742447 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:09.742489 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:09.742561 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:09.743057 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:09.743058 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:09.743061 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:09.743139 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:09.743142 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.744611 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.744611 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:09.745945 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:09.745947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.746858 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.746864 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:09.746934 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:09.746937 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:16:09.747054 (+ 117us) client_negotiation.cc:240] Negotiation successful
0924 21:16:09.747097 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":14,"thread_start_us":96,"threads_started":1}
I20260924 21:16:09.747426 26051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:09.742440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57692 (local address 127.6.210.131:43063)
0924 21:16:09.742853 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:09.742858 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:09.742869 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:16:09.742909 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:09.742911 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:09.742926 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:09.743037 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:09.743350 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.744449 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.746041 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.746789 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.746845 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:09.746900 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:09.747284 (+ 384us) server_negotiation.cc:299] Negotiation successful
0924 21:16:09.747339 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":95,"threads_started":1}
I20260924 21:16:09.747529 25712 tablet_service.cc:1467] Processing CreateTablet for tablet 35407c2293894443aa1af69b5e5b8374 (DEFAULT_TABLE table=test-table [id=7b2d7594ca5943ec924065f7544f3bdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:09.747921 25712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35407c2293894443aa1af69b5e5b8374. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:09.748178 26050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:09.742441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:40167 (local address 127.0.0.1:40990)
0924 21:16:09.742685 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:09.742693 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:09.742732 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:09.743222 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:09.743222 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:09.743224 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:09.743256 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:09.743256 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.744372 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.744373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:09.745945 (+ 1572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:09.745947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.747921 (+ 1974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.747922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:09.747987 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:09.747991 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:16:09.748046 (+ 55us) client_negotiation.cc:240] Negotiation successful
0924 21:16:09.748085 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":130,"thread_start_us":79,"threads_started":1}
I20260924 21:16:09.748134 25974 tablet_service.cc:1467] Processing CreateTablet for tablet 35407c2293894443aa1af69b5e5b8374 (DEFAULT_TABLE table=test-table [id=7b2d7594ca5943ec924065f7544f3bdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:09.748452 26053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:09.742847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40990 (local address 127.6.210.130:40167)
0924 21:16:09.743039 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:09.743044 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:09.743055 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:16:09.743099 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:09.743101 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:09.743119 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:09.743215 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:09.743320 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.744278 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.746930 (+ 2652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:09.747739 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:09.747804 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:09.747885 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:09.748323 (+ 438us) server_negotiation.cc:299] Negotiation successful
0924 21:16:09.748368 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":119,"threads_started":1}
I20260924 21:16:09.748493 25974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35407c2293894443aa1af69b5e5b8374. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:09.749567 25843 tablet_service.cc:1467] Processing CreateTablet for tablet 35407c2293894443aa1af69b5e5b8374 (DEFAULT_TABLE table=test-table [id=7b2d7594ca5943ec924065f7544f3bdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:09.749945 25843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35407c2293894443aa1af69b5e5b8374. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:09.751806 26054 tablet_bootstrap.cc:492] T 35407c2293894443aa1af69b5e5b8374 P 335fffa14632475aacf6724758730fad: Bootstrap starting.
I20260924 21:16:09.752161 26055 tablet_bootstrap.cc:492] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db: Bootstrap starting.
I20260924 21:16:09.752796 26055 tablet_bootstrap.cc:654] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:09.752796 26054 tablet_bootstrap.cc:654] T 35407c2293894443aa1af69b5e5b8374 P 335fffa14632475aacf6724758730fad: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:09.753137 26055 log.cc:824] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:09.753147 26054 log.cc:824] T 35407c2293894443aa1af69b5e5b8374 P 335fffa14632475aacf6724758730fad: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:09.754109 26054 tablet_bootstrap.cc:492] T 35407c2293894443aa1af69b5e5b8374 P 335fffa14632475aacf6724758730fad: No bootstrap required, opened a new log
I20260924 21:16:09.754195 26054 ts_tablet_manager.cc:1397] T 35407c2293894443aa1af69b5e5b8374 P 335fffa14632475aacf6724758730fad: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:16:09.754249 26055 tablet_bootstrap.cc:492] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db: No bootstrap required, opened a new log
I20260924 21:16:09.754357 26055 ts_tablet_manager.cc:1397] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:16:09.754386 26058 tablet_bootstrap.cc:492] T 35407c2293894443aa1af69b5e5b8374 P 095c821f2a9c44a2b1fb2cfe40632f2f: Bootstrap starting.
I20260924 21:16:09.755095 26058 tablet_bootstrap.cc:654] T 35407c2293894443aa1af69b5e5b8374 P 095c821f2a9c44a2b1fb2cfe40632f2f: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:09.755532 26058 log.cc:824] T 35407c2293894443aa1af69b5e5b8374 P 095c821f2a9c44a2b1fb2cfe40632f2f: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:09.757252 26055 raft_consensus.cc:348] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfb96de7e9164b05bf8407d1343063db" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 43063 } } peers { permanent_uuid: "335fffa14632475aacf6724758730fad" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 40597 } } peers { permanent_uuid: "095c821f2a9c44a2b1fb2cfe40632f2f" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40167 } }
I20260924 21:16:09.757252 26054 raft_consensus.cc:348] T 35407c2293894443aa1af69b5e5b8374 P 335fffa14632475aacf6724758730fad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfb96de7e9164b05bf8407d1343063db" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 43063 } } peers { permanent_uuid: "335fffa14632475aacf6724758730fad" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 40597 } } peers { permanent_uuid: "095c821f2a9c44a2b1fb2cfe40632f2f" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40167 } }
I20260924 21:16:09.757441 26055 raft_consensus.cc:374] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:09.757441 26054 raft_consensus.cc:374] T 35407c2293894443aa1af69b5e5b8374 P 335fffa14632475aacf6724758730fad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:09.757483 26054 raft_consensus.cc:729] T 35407c2293894443aa1af69b5e5b8374 P 335fffa14632475aacf6724758730fad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 335fffa14632475aacf6724758730fad, State: Initialized, Role: FOLLOWER
I20260924 21:16:09.757483 26055 raft_consensus.cc:729] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfb96de7e9164b05bf8407d1343063db, State: Initialized, Role: FOLLOWER
I20260924 21:16:09.757623 26055 consensus_queue.cc:260] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfb96de7e9164b05bf8407d1343063db" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 43063 } } peers { permanent_uuid: "335fffa14632475aacf6724758730fad" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 40597 } } peers { permanent_uuid: "095c821f2a9c44a2b1fb2cfe40632f2f" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40167 } }
I20260924 21:16:09.757623 26054 consensus_queue.cc:260] T 35407c2293894443aa1af69b5e5b8374 P 335fffa14632475aacf6724758730fad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfb96de7e9164b05bf8407d1343063db" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 43063 } } peers { permanent_uuid: "335fffa14632475aacf6724758730fad" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 40597 } } peers { permanent_uuid: "095c821f2a9c44a2b1fb2cfe40632f2f" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40167 } }
I20260924 21:16:09.757916 26054 ts_tablet_manager.cc:1428] T 35407c2293894443aa1af69b5e5b8374 P 335fffa14632475aacf6724758730fad: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260924 21:16:09.757916 26055 ts_tablet_manager.cc:1428] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260924 21:16:09.758067 25778 heartbeater.cc:503] Master 127.6.210.190:37681 was elected leader, sending a full tablet report...
I20260924 21:16:09.758111 26040 heartbeater.cc:503] Master 127.6.210.190:37681 was elected leader, sending a full tablet report...
I20260924 21:16:09.758783 26058 tablet_bootstrap.cc:492] T 35407c2293894443aa1af69b5e5b8374 P 095c821f2a9c44a2b1fb2cfe40632f2f: No bootstrap required, opened a new log
I20260924 21:16:09.758863 26058 ts_tablet_manager.cc:1397] T 35407c2293894443aa1af69b5e5b8374 P 095c821f2a9c44a2b1fb2cfe40632f2f: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260924 21:16:09.761209 26058 raft_consensus.cc:348] T 35407c2293894443aa1af69b5e5b8374 P 095c821f2a9c44a2b1fb2cfe40632f2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfb96de7e9164b05bf8407d1343063db" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 43063 } } peers { permanent_uuid: "335fffa14632475aacf6724758730fad" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 40597 } } peers { permanent_uuid: "095c821f2a9c44a2b1fb2cfe40632f2f" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40167 } }
I20260924 21:16:09.761348 26058 raft_consensus.cc:374] T 35407c2293894443aa1af69b5e5b8374 P 095c821f2a9c44a2b1fb2cfe40632f2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:09.761372 26058 raft_consensus.cc:729] T 35407c2293894443aa1af69b5e5b8374 P 095c821f2a9c44a2b1fb2cfe40632f2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 095c821f2a9c44a2b1fb2cfe40632f2f, State: Initialized, Role: FOLLOWER
I20260924 21:16:09.761507 26058 consensus_queue.cc:260] T 35407c2293894443aa1af69b5e5b8374 P 095c821f2a9c44a2b1fb2cfe40632f2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfb96de7e9164b05bf8407d1343063db" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 43063 } } peers { permanent_uuid: "335fffa14632475aacf6724758730fad" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 40597 } } peers { permanent_uuid: "095c821f2a9c44a2b1fb2cfe40632f2f" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40167 } }
I20260924 21:16:09.761742 26058 ts_tablet_manager.cc:1428] T 35407c2293894443aa1af69b5e5b8374 P 095c821f2a9c44a2b1fb2cfe40632f2f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:16:09.761837 25909 heartbeater.cc:503] Master 127.6.210.190:37681 was elected leader, sending a full tablet report...
W20260924 21:16:09.832830 25910 tablet.cc:2369] T 35407c2293894443aa1af69b5e5b8374 P 095c821f2a9c44a2b1fb2cfe40632f2f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 21:16:09.955541 25779 tablet.cc:2369] T 35407c2293894443aa1af69b5e5b8374 P 335fffa14632475aacf6724758730fad: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 21:16:09.966507 26041 tablet.cc:2369] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:16:10.001711 26060 raft_consensus.cc:482] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:16:10.001889 26060 raft_consensus.cc:504] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfb96de7e9164b05bf8407d1343063db" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 43063 } } peers { permanent_uuid: "335fffa14632475aacf6724758730fad" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 40597 } } peers { permanent_uuid: "095c821f2a9c44a2b1fb2cfe40632f2f" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40167 } }
I20260924 21:16:10.002312 26060 leader_election.cc:288] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 335fffa14632475aacf6724758730fad (127.6.210.129:40597), 095c821f2a9c44a2b1fb2cfe40632f2f (127.6.210.130:40167)
I20260924 21:16:10.006361 26053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:10.002745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:42089 (local address 127.6.210.130:40167)
0924 21:16:10.002815 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:10.002817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:10.002954 (+ 137us) server_negotiation.cc:407] Connection header received
0924 21:16:10.003008 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:10.003009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:10.003018 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:10.003143 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:10.003261 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:10.003922 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:10.005293 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:10.005832 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:10.005911 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:10.005920 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:10.006196 (+ 276us) server_negotiation.cc:299] Negotiation successful
0924 21:16:10.006296 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260924 21:16:10.006361 26052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:10.002622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:34577 (local address 127.6.210.129:40597)
0924 21:16:10.002706 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:10.002708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:10.002719 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:16:10.002749 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:10.002750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:10.002783 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:10.002861 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:10.003077 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:10.003848 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:10.005293 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:10.005832 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:10.005911 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:10.005920 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:10.006196 (+ 276us) server_negotiation.cc:299] Negotiation successful
0924 21:16:10.006250 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260924 21:16:10.006253 26042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:10.002526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:40597 (local address 127.6.210.131:34577)
0924 21:16:10.002585 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:10.002598 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:10.002648 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:10.002914 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:10.002915 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:10.002916 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:10.002989 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:10.002991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:10.003934 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:10.003934 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:10.005190 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:10.005192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:10.005924 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:10.005925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:10.005999 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:10.006003 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:16:10.006064 (+ 61us) client_negotiation.cc:240] Negotiation successful
0924 21:16:10.006113 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":16}
I20260924 21:16:10.006282 26063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:10.002663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:40167 (local address 127.6.210.131:42089)
0924 21:16:10.002835 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:10.002889 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:10.002945 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:10.003158 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:10.003159 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:10.003160 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:10.003205 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:10.003206 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:10.003998 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:10.003998 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:10.005197 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:10.005199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:10.005925 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:10.005926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:10.006004 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:10.006007 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:16:10.006064 (+ 57us) client_negotiation.cc:240] Negotiation successful
0924 21:16:10.006107 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":2,"thread_start_us":106,"threads_started":1}
I20260924 21:16:10.006832 25732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35407c2293894443aa1af69b5e5b8374" candidate_uuid: "dfb96de7e9164b05bf8407d1343063db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "335fffa14632475aacf6724758730fad" is_pre_election: true
I20260924 21:16:10.006961 25732 raft_consensus.cc:2454] T 35407c2293894443aa1af69b5e5b8374 P 335fffa14632475aacf6724758730fad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfb96de7e9164b05bf8407d1343063db in term 0.
I20260924 21:16:10.007047 25863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35407c2293894443aa1af69b5e5b8374" candidate_uuid: "dfb96de7e9164b05bf8407d1343063db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "095c821f2a9c44a2b1fb2cfe40632f2f" is_pre_election: true
I20260924 21:16:10.007228 25863 raft_consensus.cc:2454] T 35407c2293894443aa1af69b5e5b8374 P 095c821f2a9c44a2b1fb2cfe40632f2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfb96de7e9164b05bf8407d1343063db in term 0.
I20260924 21:16:10.007282 25929 leader_election.cc:302] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 335fffa14632475aacf6724758730fad, dfb96de7e9164b05bf8407d1343063db; no voters:
I20260924 21:16:10.007406 26060 raft_consensus.cc:2781] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:16:10.007463 26060 raft_consensus.cc:482] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:16:10.007496 26060 raft_consensus.cc:3037] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:10.008191 26060 raft_consensus.cc:504] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfb96de7e9164b05bf8407d1343063db" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 43063 } } peers { permanent_uuid: "335fffa14632475aacf6724758730fad" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 40597 } } peers { permanent_uuid: "095c821f2a9c44a2b1fb2cfe40632f2f" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40167 } }
I20260924 21:16:10.008328 26060 leader_election.cc:288] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db [CANDIDATE]: Term 1 election: Requested vote from peers 335fffa14632475aacf6724758730fad (127.6.210.129:40597), 095c821f2a9c44a2b1fb2cfe40632f2f (127.6.210.130:40167)
I20260924 21:16:10.008496 25732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35407c2293894443aa1af69b5e5b8374" candidate_uuid: "dfb96de7e9164b05bf8407d1343063db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "335fffa14632475aacf6724758730fad"
I20260924 21:16:10.008526 25863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35407c2293894443aa1af69b5e5b8374" candidate_uuid: "dfb96de7e9164b05bf8407d1343063db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "095c821f2a9c44a2b1fb2cfe40632f2f"
I20260924 21:16:10.008605 25863 raft_consensus.cc:3037] T 35407c2293894443aa1af69b5e5b8374 P 095c821f2a9c44a2b1fb2cfe40632f2f [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:10.008606 25732 raft_consensus.cc:3037] T 35407c2293894443aa1af69b5e5b8374 P 335fffa14632475aacf6724758730fad [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:10.009511 25863 raft_consensus.cc:2454] T 35407c2293894443aa1af69b5e5b8374 P 095c821f2a9c44a2b1fb2cfe40632f2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfb96de7e9164b05bf8407d1343063db in term 1.
I20260924 21:16:10.009521 25732 raft_consensus.cc:2454] T 35407c2293894443aa1af69b5e5b8374 P 335fffa14632475aacf6724758730fad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfb96de7e9164b05bf8407d1343063db in term 1.
I20260924 21:16:10.009696 25929 leader_election.cc:302] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 095c821f2a9c44a2b1fb2cfe40632f2f, dfb96de7e9164b05bf8407d1343063db; no voters:
I20260924 21:16:10.009804 26060 raft_consensus.cc:2781] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:10.009958 26060 raft_consensus.cc:686] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db [term 1 LEADER]: Becoming Leader. State: Replica: dfb96de7e9164b05bf8407d1343063db, State: Running, Role: LEADER
I20260924 21:16:10.010061 26060 consensus_queue.cc:237] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfb96de7e9164b05bf8407d1343063db" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 43063 } } peers { permanent_uuid: "335fffa14632475aacf6724758730fad" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 40597 } } peers { permanent_uuid: "095c821f2a9c44a2b1fb2cfe40632f2f" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40167 } }
I20260924 21:16:10.010937 25590 catalog_manager.cc:5495] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db reported cstate change: term changed from 0 to 1, leader changed from <none> to dfb96de7e9164b05bf8407d1343063db (127.6.210.131). New cstate: current_term: 1 leader_uuid: "dfb96de7e9164b05bf8407d1343063db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfb96de7e9164b05bf8407d1343063db" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 43063 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "335fffa14632475aacf6724758730fad" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 40597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "095c821f2a9c44a2b1fb2cfe40632f2f" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40167 } health_report { overall_health: UNKNOWN } } }
I20260924 21:16:10.034037 26051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:10.031582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57704 (local address 127.6.210.131:43063)
0924 21:16:10.031652 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:10.031654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:10.031665 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:16:10.031729 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:10.031730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:10.031751 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:10.031831 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:16:10.031973 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:10.032465 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:10.033255 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:10.033444 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:10.033525 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:10.033675 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:16:10.033779 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:16:10.033891 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:10.033922 (+ 31us) server_negotiation.cc:299] Negotiation successful
0924 21:16:10.033932 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 21:16:10.036022 25863 raft_consensus.cc:1261] T 35407c2293894443aa1af69b5e5b8374 P 095c821f2a9c44a2b1fb2cfe40632f2f [term 1 FOLLOWER]: Refusing update from remote peer dfb96de7e9164b05bf8407d1343063db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:16:10.036046 25732 raft_consensus.cc:1261] T 35407c2293894443aa1af69b5e5b8374 P 335fffa14632475aacf6724758730fad [term 1 FOLLOWER]: Refusing update from remote peer dfb96de7e9164b05bf8407d1343063db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:16:10.036473 26060 consensus_queue.cc:1035] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db [LEADER]: Connected to new peer: Peer: permanent_uuid: "335fffa14632475aacf6724758730fad" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 40597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:16:10.036599 26064 consensus_queue.cc:1035] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db [LEADER]: Connected to new peer: Peer: permanent_uuid: "095c821f2a9c44a2b1fb2cfe40632f2f" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 40167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:16:10.038870 26067 mvcc.cc:205] Tried to move back new op lower bound from 7333005598864568320 to 7333005598761799680. Current Snapshot: MvccSnapshot[applied={T|T < 7333005598864568320}]
I20260924 21:16:10.039989 26069 mvcc.cc:205] Tried to move back new op lower bound from 7333005598864568320 to 7333005598761799680. Current Snapshot: MvccSnapshot[applied={T|T < 7333005598864568320}]
I20260924 21:16:10.041457 25588 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:16:10.041536 25588 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:16:10.042264 25588 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 589a3dbb24ae4cdc91fa48c1814bcc6f: Sending DeleteTablet for 3 replicas of tablet 35407c2293894443aa1af69b5e5b8374
I20260924 21:16:10.042577 25974 tablet_service.cc:1514] Processing DeleteTablet for tablet 35407c2293894443aa1af69b5e5b8374 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:10 UTC) from {username='slave'} at 127.0.0.1:57692
I20260924 21:16:10.042668 25843 tablet_service.cc:1514] Processing DeleteTablet for tablet 35407c2293894443aa1af69b5e5b8374 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:10 UTC) from {username='slave'} at 127.0.0.1:40990
I20260924 21:16:10.042668 25712 tablet_service.cc:1514] Processing DeleteTablet for tablet 35407c2293894443aa1af69b5e5b8374 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:10 UTC) from {username='slave'} at 127.0.0.1:41244
I20260924 21:16:10.043064 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 25650
W20260924 21:16:10.048928 25576 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.210.129:40597 (error 108)
I20260924 21:16:10.048992 26077 tablet_replica.cc:330] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db: stopping tablet replica
I20260924 21:16:10.049094 26077 raft_consensus.cc:2229] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:16:10.049224 26077 raft_consensus.cc:2258] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:16:10.049224 26079 tablet_replica.cc:330] T 35407c2293894443aa1af69b5e5b8374 P 095c821f2a9c44a2b1fb2cfe40632f2f: stopping tablet replica
I20260924 21:16:10.049470 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 25781
I20260924 21:16:10.050495 26077 ts_tablet_manager.cc:1905] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:16:10.050633 26050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:10.050507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:40597 (local address 127.0.0.1:41258)
0924 21:16:10.050556 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:10.050593 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.129:40597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260924 21:16:10.050748 25576 catalog_manager.cc:4584] TS 335fffa14632475aacf6724758730fad (127.6.210.129:40597): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 35407c2293894443aa1af69b5e5b8374: Network error: Client connection negotiation failed: client connection to 127.6.210.129:40597: connect: Connection refused (error 111)
I20260924 21:16:10.052085 26077 ts_tablet_manager.cc:1918] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:16:10.052165 26077 log.cc:1196] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284333163253-6986-0/minicluster-data/ts-2/wal/wals/35407c2293894443aa1af69b5e5b8374
I20260924 21:16:10.052418 26077 ts_tablet_manager.cc:1939] T 35407c2293894443aa1af69b5e5b8374 P dfb96de7e9164b05bf8407d1343063db: Deleting consensus metadata
I20260924 21:16:10.052877 25574 catalog_manager.cc:4841] TS dfb96de7e9164b05bf8407d1343063db (127.6.210.131:43063): tablet 35407c2293894443aa1af69b5e5b8374 (table test-table [id=7b2d7594ca5943ec924065f7544f3bdd]) successfully deleted
W20260924 21:16:10.055400 26050 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:16:10.055267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:40167 (local address 127.0.0.1:41000)
0924 21:16:10.055320 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:10.055346 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.130:40167: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":34}
I20260924 21:16:10.055660 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 25912
I20260924 21:16:10.060988 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 25559
2026-09-24T21:16:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (911 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:16:10 dist-test-slave-q276 krb5kdc[26083](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:16:10 dist-test-slave-q276 krb5kdc[26083](info): set up 2 sockets
Sep 24 21:16:10 dist-test-slave-q276 krb5kdc[26083](info): commencing operation
krb5kdc: starting...
W20260924 21:16:11.106601 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:16:11 dist-test-slave-q276 krb5kdc[26083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:16:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:16:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.210.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
I20260924 21:16:11.225669 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:41439
--webserver_interface=127.6.210.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:39413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:41439
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:11.310657 26096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:11.310906 26096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:11.310942 26096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:11.312112 26096 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:11.312181 26096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:11.312206 26096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:11.312227 26096 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:11.312254 26096 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:11.313776 26096 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:39413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:41439
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:41439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:11.314046 26096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:11.314298 26096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:11.319257 26102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:11.319257 26104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:11.319373 26101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:11.319591 26096 server_base.cc:1034] running on GCE node
I20260924 21:16:11.319906 26096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:11.320241 26096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:11.321405 26096 hybrid_clock.cc:648] HybridClock initialized: now 1790284571321365 us; error 53 us; skew 500 ppm
Sep 24 21:16:11 dist-test-slave-q276 krb5kdc[26083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284571, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:11.324693 26096 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:16:11.325275 26096 webserver.cc:460] Webserver started at http://127.6.210.190:45375/ using document root <none> and password file <none>
I20260924 21:16:11.325531 26096 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:11.325606 26096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:11.325774 26096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:11.327077 26096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "48b89b8ccec04af2886e87106460f1cd"
format_stamp: "Formatted at 2026-09-24 21:16:11 on dist-test-slave-q276"
server_key: "f1ca7134347d011cea515ff6552eba90"
server_key_iv: "fd8f379048d0af71b1ee3b800b71fa15"
server_key_version: "encryptionkey@0"
I20260924 21:16:11.327486 26096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "48b89b8ccec04af2886e87106460f1cd"
format_stamp: "Formatted at 2026-09-24 21:16:11 on dist-test-slave-q276"
server_key: "f1ca7134347d011cea515ff6552eba90"
server_key_iv: "fd8f379048d0af71b1ee3b800b71fa15"
server_key_version: "encryptionkey@0"
I20260924 21:16:11.328944 26096 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:16:11.329777 26111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:11.329990 26096 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:11.330104 26096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "48b89b8ccec04af2886e87106460f1cd"
format_stamp: "Formatted at 2026-09-24 21:16:11 on dist-test-slave-q276"
server_key: "f1ca7134347d011cea515ff6552eba90"
server_key_iv: "fd8f379048d0af71b1ee3b800b71fa15"
server_key_version: "encryptionkey@0"
I20260924 21:16:11.330193 26096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:11.361557 26096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:11.363921 26096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:11.364121 26096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:11.368372 26096 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:41439
I20260924 21:16:11.368441 26163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:41439 every 8 connection(s)
I20260924 21:16:11.368819 26096 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:16:11.369767 26164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:11.371179 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 26096
I20260924 21:16:11.371299 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:16:11.371520 6986 external_mini_cluster.cc:1428] Setting key dbe05b1e1e572b36c07b75dc7f0490ba
I20260924 21:16:11.373077 26164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48b89b8ccec04af2886e87106460f1cd: Bootstrap starting.
I20260924 21:16:11.373931 26164 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48b89b8ccec04af2886e87106460f1cd: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:11.374312 26164 log.cc:824] T 00000000000000000000000000000000 P 48b89b8ccec04af2886e87106460f1cd: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:11.375290 26164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48b89b8ccec04af2886e87106460f1cd: No bootstrap required, opened a new log
I20260924 21:16:11.378013 26164 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48b89b8ccec04af2886e87106460f1cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48b89b8ccec04af2886e87106460f1cd" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 41439 } }
I20260924 21:16:11.378243 26164 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48b89b8ccec04af2886e87106460f1cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:11.378319 26164 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48b89b8ccec04af2886e87106460f1cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48b89b8ccec04af2886e87106460f1cd, State: Initialized, Role: FOLLOWER
I20260924 21:16:11.378471 26164 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48b89b8ccec04af2886e87106460f1cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48b89b8ccec04af2886e87106460f1cd" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 41439 } }
I20260924 21:16:11.378577 26164 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48b89b8ccec04af2886e87106460f1cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:16:11.378616 26164 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48b89b8ccec04af2886e87106460f1cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:16:11.378672 26164 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48b89b8ccec04af2886e87106460f1cd [term 0 FOLLOWER]: Advancing to term 1
Sep 24 21:16:11 dist-test-slave-q276 krb5kdc[26083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:16:11.379518 26164 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48b89b8ccec04af2886e87106460f1cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48b89b8ccec04af2886e87106460f1cd" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 41439 } }
I20260924 21:16:11.379650 26164 leader_election.cc:302] T 00000000000000000000000000000000 P 48b89b8ccec04af2886e87106460f1cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48b89b8ccec04af2886e87106460f1cd; no voters:
I20260924 21:16:11.379840 26164 leader_election.cc:288] T 00000000000000000000000000000000 P 48b89b8ccec04af2886e87106460f1cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:16:11.379900 26169 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48b89b8ccec04af2886e87106460f1cd [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:11.380069 26169 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48b89b8ccec04af2886e87106460f1cd [term 1 LEADER]: Becoming Leader. State: Replica: 48b89b8ccec04af2886e87106460f1cd, State: Running, Role: LEADER
I20260924 21:16:11.380138 26164 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48b89b8ccec04af2886e87106460f1cd [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:11.380218 26169 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48b89b8ccec04af2886e87106460f1cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48b89b8ccec04af2886e87106460f1cd" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 41439 } }
I20260924 21:16:11.380753 26170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48b89b8ccec04af2886e87106460f1cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48b89b8ccec04af2886e87106460f1cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48b89b8ccec04af2886e87106460f1cd" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 41439 } } }
I20260924 21:16:11.380568 26171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48b89b8ccec04af2886e87106460f1cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48b89b8ccec04af2886e87106460f1cd. Latest consensus state: current_term: 1 leader_uuid: "48b89b8ccec04af2886e87106460f1cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48b89b8ccec04af2886e87106460f1cd" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 41439 } } }
I20260924 21:16:11.381021 26171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48b89b8ccec04af2886e87106460f1cd [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:11.380923 26170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48b89b8ccec04af2886e87106460f1cd [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:11.381614 26179 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:16:11.382128 26179 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:16:11.383890 26179 catalog_manager.cc:1347] Generated new cluster ID: a4d73b524d5a42159d866a823fa6a49c
I20260924 21:16:11.383939 26179 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:16:11.384557 26167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:11.372069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41180 (local address 127.6.210.190:41439)
0924 21:16:11.372250 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:11.372258 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:11.372283 (+ 25us) server_negotiation.cc:407] Connection header received
0924 21:16:11.372805 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:11.372812 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:11.372863 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:11.372980 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:11.373896 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:11.375093 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:11.376280 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:11.376691 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:11.376770 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:11.379995 (+ 3225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:11.380000 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:11.380000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:11.380014 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:11.382329 (+ 2315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:11.383165 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:11.383167 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:11.383168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:11.383258 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:11.383524 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:11.383525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:11.383525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:11.383858 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:11.384013 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:11.384182 (+ 169us) server_negotiation.cc:299] Negotiation successful
0924 21:16:11.384365 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":87,"threads_started":1}
I20260924 21:16:11.395714 26179 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:16:11.396410 26179 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:16:11.402179 26179 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48b89b8ccec04af2886e87106460f1cd: Generated new TSK 0
I20260924 21:16:11.402446 26179 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.210.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
Entry for principal kudu/127.6.210.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab.
I20260924 21:16:11.436452 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.129:0
--local_ip_for_outbound_sockets=127.6.210.129
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:41439
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:39413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:11.536015 26190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:11.536264 26190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:11.536307 26190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:11.537456 26190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:11.537525 26190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:11.537569 26190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.129
I20260924 21:16:11.539119 26190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:39413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.129.keytab
--principal=kudu/127.6.210.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:41439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.129
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:11.539389 26190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:11.539623 26190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:11.544646 26198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:11.544718 26195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:11.544657 26196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:11.545300 26190 server_base.cc:1034] running on GCE node
I20260924 21:16:11.545516 26190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:11.545783 26190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:11.546943 26190 hybrid_clock.cc:648] HybridClock initialized: now 1790284571546931 us; error 28 us; skew 500 ppm
Sep 24 21:16:11 dist-test-slave-q276 krb5kdc[26083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284571, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:11.550170 26190 init.cc:377] Logged in from keytab as kudu/127.6.210.129@KRBTEST.COM (short username kudu)
I20260924 21:16:11.550752 26190 webserver.cc:460] Webserver started at http://127.6.210.129:42669/ using document root <none> and password file <none>
I20260924 21:16:11.550981 26190 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:11.551052 26190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:11.551218 26190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:11.552290 26190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-0/data/instance:
uuid: "b0ceaf06e9ff4aa4bdf3c974e9217b10"
format_stamp: "Formatted at 2026-09-24 21:16:11 on dist-test-slave-q276"
server_key: "17c53f9767919ca05fed836983dc88ae"
server_key_iv: "fa50ecf963037e75823309cb9f9485e4"
server_key_version: "encryptionkey@0"
I20260924 21:16:11.552654 26190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance:
uuid: "b0ceaf06e9ff4aa4bdf3c974e9217b10"
format_stamp: "Formatted at 2026-09-24 21:16:11 on dist-test-slave-q276"
server_key: "17c53f9767919ca05fed836983dc88ae"
server_key_iv: "fa50ecf963037e75823309cb9f9485e4"
server_key_version: "encryptionkey@0"
I20260924 21:16:11.554044 26190 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260924 21:16:11.555063 26205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:11.555331 26190 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:11.555449 26190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-0/wal
uuid: "b0ceaf06e9ff4aa4bdf3c974e9217b10"
format_stamp: "Formatted at 2026-09-24 21:16:11 on dist-test-slave-q276"
server_key: "17c53f9767919ca05fed836983dc88ae"
server_key_iv: "fa50ecf963037e75823309cb9f9485e4"
server_key_version: "encryptionkey@0"
I20260924 21:16:11.555544 26190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:11.568898 26190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:11.570938 26190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:11.571116 26190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:11.571375 26190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:16:11.571691 26190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:16:11.571725 26190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:11.571755 26190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:16:11.571796 26190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:11.577638 26190 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.129:34473
I20260924 21:16:11.577688 26318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.129:34473 every 8 connection(s)
I20260924 21:16:11.578008 26190 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
I20260924 21:16:11.582939 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 26190
I20260924 21:16:11.583073 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance
I20260924 21:16:11.583284 6986 external_mini_cluster.cc:1428] Setting key 3def15bd4dbbb68a75c7a943a9f6a284
Sep 24 21:16:11 dist-test-slave-q276 krb5kdc[26083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284571, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.129@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:16:11.589313 26321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:11.578790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:41439 (local address 127.6.210.129:59085)
0924 21:16:11.578982 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:11.579021 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:11.579602 (+ 581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:11.579883 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:11.579887 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:11.580667 (+ 780us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:11.580892 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:11.580896 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:11.581890 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:11.581892 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:11.582973 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:11.582976 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:11.583445 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:11.583447 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:11.583584 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:11.584211 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:11.584221 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:11.585802 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:11.587888 (+ 2086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:11.587895 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:11.587898 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:11.588133 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:11.588367 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:11.588367 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:11.588368 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:11.588468 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:11.588784 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:11.588788 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:11.588910 (+ 122us) client_negotiation.cc:769] Sending connection context
0924 21:16:11.588997 (+ 87us) client_negotiation.cc:240] Negotiation successful
0924 21:16:11.589123 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":89,"threads_started":1}
I20260924 21:16:11.589329 26167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:11.578814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:59085 (local address 127.6.210.190:41439)
0924 21:16:11.578887 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:11.578889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:11.579112 (+ 223us) server_negotiation.cc:407] Connection header received
0924 21:16:11.579724 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:11.579725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:11.579738 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:11.579803 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:11.581084 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:11.581787 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:11.583080 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:11.583343 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:11.583439 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:11.585898 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:11.585900 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:11.585900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:11.585916 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:11.587788 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:11.588214 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:11.588216 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:11.588216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:11.588288 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:11.588538 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:11.588539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:11.588540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:11.588707 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:11.588794 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:11.589052 (+ 258us) server_negotiation.cc:299] Negotiation successful
0924 21:16:11.589174 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 21:16:11.590025 26319 heartbeater.cc:347] Connected to a master server at 127.6.210.190:41439
I20260924 21:16:11.590121 26319 heartbeater.cc:464] Registering TS with master...
I20260924 21:16:11.590366 26319 heartbeater.cc:511] Master 127.6.210.190:41439 requested a full tablet report, sending...
I20260924 21:16:11.590898 26128 ts_manager.cc:194] Registered new tserver with Master: b0ceaf06e9ff4aa4bdf3c974e9217b10 (127.6.210.129:34473)
I20260924 21:16:11.591521 26128 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.129@KRBTEST.COM'} at 127.6.210.129:59085
WARNING: no policy specified for kudu/127.6.210.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
Entry for principal kudu/127.6.210.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab.
I20260924 21:16:11.613433 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.130:0
--local_ip_for_outbound_sockets=127.6.210.130
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:41439
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:39413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:11.695930 26324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:11.696172 26324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:11.696213 26324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:11.697469 26324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:11.697535 26324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:11.697577 26324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.130
I20260924 21:16:11.699087 26324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:39413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.130.keytab
--principal=kudu/127.6.210.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:41439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.130
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:11.699362 26324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:11.699589 26324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:11.705201 26332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:11.705399 26329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:11.705477 26330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:11.705977 26324 server_base.cc:1034] running on GCE node
I20260924 21:16:11.706175 26324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:11.706476 26324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:11.707645 26324 hybrid_clock.cc:648] HybridClock initialized: now 1790284571707635 us; error 32 us; skew 500 ppm
Sep 24 21:16:11 dist-test-slave-q276 krb5kdc[26083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284571, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:11.710191 26324 init.cc:377] Logged in from keytab as kudu/127.6.210.130@KRBTEST.COM (short username kudu)
I20260924 21:16:11.710647 26324 webserver.cc:460] Webserver started at http://127.6.210.130:38319/ using document root <none> and password file <none>
I20260924 21:16:11.710882 26324 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:11.710951 26324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:11.711103 26324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:11.712182 26324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-1/data/instance:
uuid: "22a9294bc5e649ce90ee61cc677b3e67"
format_stamp: "Formatted at 2026-09-24 21:16:11 on dist-test-slave-q276"
server_key: "1871fb961af7693fa850c26e6cc96766"
server_key_iv: "e47b046205a240d38f1c7f3f549ceb10"
server_key_version: "encryptionkey@0"
I20260924 21:16:11.712564 26324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance:
uuid: "22a9294bc5e649ce90ee61cc677b3e67"
format_stamp: "Formatted at 2026-09-24 21:16:11 on dist-test-slave-q276"
server_key: "1871fb961af7693fa850c26e6cc96766"
server_key_iv: "e47b046205a240d38f1c7f3f549ceb10"
server_key_version: "encryptionkey@0"
I20260924 21:16:11.713994 26324 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:16:11.714814 26339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:11.714988 26324 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:11.715087 26324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-1/wal
uuid: "22a9294bc5e649ce90ee61cc677b3e67"
format_stamp: "Formatted at 2026-09-24 21:16:11 on dist-test-slave-q276"
server_key: "1871fb961af7693fa850c26e6cc96766"
server_key_iv: "e47b046205a240d38f1c7f3f549ceb10"
server_key_version: "encryptionkey@0"
I20260924 21:16:11.715173 26324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:11.721788 26324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:11.723752 26324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:11.723922 26324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:11.724159 26324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:16:11.724457 26324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:16:11.724510 26324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:11.724551 26324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:16:11.724577 26324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:11.730429 26324 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.130:35753
I20260924 21:16:11.730469 26452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.130:35753 every 8 connection(s)
I20260924 21:16:11.730818 26324 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
Sep 24 21:16:11 dist-test-slave-q276 krb5kdc[26083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284571, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.130@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:16:11.738737 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 26324
I20260924 21:16:11.738873 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance
I20260924 21:16:11.739071 6986 external_mini_cluster.cc:1428] Setting key 325bd1bc30dd4315827ae84446e34d4c
I20260924 21:16:11.740789 26167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:11.731613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:50151 (local address 127.6.210.190:41439)
0924 21:16:11.731709 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:11.731711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:11.731950 (+ 239us) server_negotiation.cc:407] Connection header received
0924 21:16:11.732475 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:11.732476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:11.732485 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:11.732555 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:11.733666 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:11.734188 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:11.735250 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:11.735457 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:11.735517 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:11.737711 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:11.737712 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:11.737712 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:11.737719 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:11.739385 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:11.739806 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:11.739808 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:11.739808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:11.739871 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:11.740117 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:11.740118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:11.740118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:11.740267 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:11.740343 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:11.740616 (+ 273us) server_negotiation.cc:299] Negotiation successful
0924 21:16:11.740694 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260924 21:16:11.741032 26455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:11.731618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:41439 (local address 127.6.210.130:50151)
0924 21:16:11.731806 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:11.731849 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:11.732377 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:11.732615 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:11.732619 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:11.733314 (+ 695us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:11.733580 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:11.733586 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:11.734333 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:11.734335 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:11.735163 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:11.735167 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:11.735558 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:11.735559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:11.735699 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:11.736187 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:11.736196 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:11.737617 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:11.739473 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:11.739480 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:11.739483 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:11.739725 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:11.739942 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:11.739943 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:11.739943 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:11.740047 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:11.740349 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:11.740356 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:11.740481 (+ 125us) client_negotiation.cc:769] Sending connection context
0924 21:16:11.740593 (+ 112us) client_negotiation.cc:240] Negotiation successful
0924 21:16:11.740764 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":115,"threads_started":1}
I20260924 21:16:11.741557 26453 heartbeater.cc:347] Connected to a master server at 127.6.210.190:41439
I20260924 21:16:11.741667 26453 heartbeater.cc:464] Registering TS with master...
I20260924 21:16:11.741863 26453 heartbeater.cc:511] Master 127.6.210.190:41439 requested a full tablet report, sending...
I20260924 21:16:11.742233 26128 ts_manager.cc:194] Registered new tserver with Master: 22a9294bc5e649ce90ee61cc677b3e67 (127.6.210.130:35753)
I20260924 21:16:11.742620 26128 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.130@KRBTEST.COM'} at 127.6.210.130:50151
WARNING: no policy specified for kudu/127.6.210.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
Entry for principal kudu/127.6.210.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab.
I20260924 21:16:11.765799 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.131:0
--local_ip_for_outbound_sockets=127.6.210.131
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:41439
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:39413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:11.878628 26458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:11.878856 26458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:11.878881 26458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:11.879989 26458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:11.880030 26458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:11.880057 26458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.131
I20260924 21:16:11.881767 26458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:39413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.131.keytab
--principal=kudu/127.6.210.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:41439
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.131
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:11.882017 26458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:11.882342 26458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:11.887368 26466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:11.887413 26458 server_base.cc:1034] running on GCE node
W20260924 21:16:11.887368 26464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:11.887393 26463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:11.887738 26458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:11.887996 26458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:11.889180 26458 hybrid_clock.cc:648] HybridClock initialized: now 1790284571889164 us; error 28 us; skew 500 ppm
Sep 24 21:16:11 dist-test-slave-q276 krb5kdc[26083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284571, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:11.891846 26458 init.cc:377] Logged in from keytab as kudu/127.6.210.131@KRBTEST.COM (short username kudu)
I20260924 21:16:11.892336 26458 webserver.cc:460] Webserver started at http://127.6.210.131:34967/ using document root <none> and password file <none>
I20260924 21:16:11.892592 26458 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:11.892642 26458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:11.892819 26458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:11.893904 26458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-2/data/instance:
uuid: "480b86b3f96e42509b520dfb58576237"
format_stamp: "Formatted at 2026-09-24 21:16:11 on dist-test-slave-q276"
server_key: "37ee162e6d5fa478eef25a56cc6e53d6"
server_key_iv: "7a4563524629c6ba8c65585c611bc786"
server_key_version: "encryptionkey@0"
I20260924 21:16:11.894353 26458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance:
uuid: "480b86b3f96e42509b520dfb58576237"
format_stamp: "Formatted at 2026-09-24 21:16:11 on dist-test-slave-q276"
server_key: "37ee162e6d5fa478eef25a56cc6e53d6"
server_key_iv: "7a4563524629c6ba8c65585c611bc786"
server_key_version: "encryptionkey@0"
I20260924 21:16:11.895709 26458 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:16:11.896488 26473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:11.896678 26458 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:11.896771 26458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-2/wal
uuid: "480b86b3f96e42509b520dfb58576237"
format_stamp: "Formatted at 2026-09-24 21:16:11 on dist-test-slave-q276"
server_key: "37ee162e6d5fa478eef25a56cc6e53d6"
server_key_iv: "7a4563524629c6ba8c65585c611bc786"
server_key_version: "encryptionkey@0"
I20260924 21:16:11.896852 26458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:11.937417 26458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:11.939723 26458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:11.939909 26458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:11.940193 26458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:16:11.940541 26458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:16:11.940601 26458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:11.940647 26458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:16:11.940696 26458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:11.947199 26458 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.131:34031
I20260924 21:16:11.947249 26586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.131:34031 every 8 connection(s)
I20260924 21:16:11.947585 26458 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
I20260924 21:16:11.952545 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 26458
I20260924 21:16:11.952662 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance
I20260924 21:16:11.952837 6986 external_mini_cluster.cc:1428] Setting key 1dc43c0447758e52c4d8707ce64479fc
Sep 24 21:16:11 dist-test-slave-q276 krb5kdc[26083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284571, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.131@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:16:11.957057 26167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:11.948285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:53049 (local address 127.6.210.190:41439)
0924 21:16:11.948360 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:11.948362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:11.948615 (+ 253us) server_negotiation.cc:407] Connection header received
0924 21:16:11.949060 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:11.949062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:11.949074 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:11.949146 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:11.950144 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:11.950772 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:11.951751 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:11.951942 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:11.952007 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:11.954564 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:11.954565 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:11.954565 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:11.954572 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:11.955808 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:11.956193 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:11.956194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:11.956194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:11.956252 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:11.956503 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:11.956504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:11.956504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:11.956622 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:11.956696 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:11.956924 (+ 228us) server_negotiation.cc:299] Negotiation successful
0924 21:16:11.956994 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260924 21:16:11.957222 26589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:11.948268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:41439 (local address 127.6.210.131:53049)
0924 21:16:11.948452 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:11.948494 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:11.948973 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:11.949249 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:11.949252 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:11.949845 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:11.950039 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:11.950043 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:11.950861 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:11.950862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:11.951655 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:11.951657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:11.952022 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:11.952023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:11.952122 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:11.952797 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:11.952806 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:11.954483 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:11.955876 (+ 1393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:11.955881 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:11.955883 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:11.956102 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:11.956320 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:11.956321 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:11.956321 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:11.956420 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:11.956689 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:11.956693 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:11.956808 (+ 115us) client_negotiation.cc:769] Sending connection context
0924 21:16:11.956891 (+ 83us) client_negotiation.cc:240] Negotiation successful
0924 21:16:11.957009 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":114,"threads_started":1}
I20260924 21:16:11.957700 26587 heartbeater.cc:347] Connected to a master server at 127.6.210.190:41439
I20260924 21:16:11.957791 26587 heartbeater.cc:464] Registering TS with master...
I20260924 21:16:11.957927 26587 heartbeater.cc:511] Master 127.6.210.190:41439 requested a full tablet report, sending...
I20260924 21:16:11.958352 26128 ts_manager.cc:194] Registered new tserver with Master: 480b86b3f96e42509b520dfb58576237 (127.6.210.131:34031)
I20260924 21:16:11.958727 26128 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.210.131@KRBTEST.COM'} at 127.6.210.131:53049
I20260924 21:16:11.964691 6986 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:16:11.972765 26167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:11.966042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41194 (local address 127.6.210.190:41439)
0924 21:16:11.966144 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:11.966147 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:11.966255 (+ 108us) server_negotiation.cc:407] Connection header received
0924 21:16:11.966391 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:11.966392 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:11.966403 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:11.966455 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:11.967096 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:11.967642 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:11.968451 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:11.968682 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:11.968775 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:11.970059 (+ 1284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:11.970060 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:11.970061 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:11.970070 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:11.971484 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:11.971891 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:11.971892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:11.971892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:11.971954 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:11.972170 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:11.972171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:11.972171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:11.972325 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:11.972427 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:11.972579 (+ 152us) server_negotiation.cc:299] Negotiation successful
0924 21:16:11.972662 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260924 21:16:11.974436 26128 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41194:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:16:11.976130 26128 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:16:11.983404 26601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:11.978722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57694 (local address 127.6.210.129:34473)
0924 21:16:11.979051 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:11.979057 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:11.979067 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:16:11.979110 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:11.979112 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:11.979129 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:11.979242 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:11.979444 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:11.980635 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:11.982238 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:11.982998 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:11.983082 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:11.983143 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:11.983282 (+ 139us) server_negotiation.cc:299] Negotiation successful
0924 21:16:11.983320 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":91,"threads_started":1}
I20260924 21:16:11.983392 26599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:11.978520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:34473 (local address 127.0.0.1:57694)
0924 21:16:11.978698 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:11.978741 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:11.978828 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:11.979259 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:11.979263 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:11.979266 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:11.979339 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:11.979341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:11.980776 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:11.980777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:11.982069 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:11.982072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:11.983088 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:11.983089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:11.983167 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:11.983182 (+ 15us) client_negotiation.cc:769] Sending connection context
0924 21:16:11.983261 (+ 79us) client_negotiation.cc:240] Negotiation successful
0924 21:16:11.983311 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":34,"thread_start_us":61,"threads_started":1}
I20260924 21:16:11.983804 26598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:11.978523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:35753 (local address 127.0.0.1:55608)
0924 21:16:11.978845 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:11.978856 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:11.978919 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:11.979340 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:11.979341 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:11.979344 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:11.979402 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:11.979404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:11.980618 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:11.980619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:11.982727 (+ 2108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:11.982730 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:11.983597 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:11.983598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:11.983667 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:11.983671 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:16:11.983700 (+ 29us) client_negotiation.cc:240] Negotiation successful
0924 21:16:11.983740 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":96,"thread_start_us":111,"threads_started":1}
I20260924 21:16:11.984014 26603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:11.978969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54780 (local address 127.6.210.131:34031)
0924 21:16:11.979151 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:11.979156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:11.979168 (+ 12us) server_negotiation.cc:407] Connection header received
0924 21:16:11.979410 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:11.979413 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:11.979435 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:11.979533 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:11.979611 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:11.980702 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:11.982238 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:11.982998 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:11.983082 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:11.983143 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:11.983862 (+ 719us) server_negotiation.cc:299] Negotiation successful
0924 21:16:11.983936 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":114,"threads_started":1}
I20260924 21:16:11.984120 26602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:11.978745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:34031 (local address 127.0.0.1:54780)
0924 21:16:11.978902 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:11.978911 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:11.978951 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:11.979527 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:11.979528 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:11.979531 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:11.979561 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:11.979562 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:11.980801 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:11.980802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:11.982095 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:11.982097 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:11.983674 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:11.983675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:11.983746 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:11.983749 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:16:11.983842 (+ 93us) client_negotiation.cc:240] Negotiation successful
0924 21:16:11.983998 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":56,"thread_start_us":88,"threads_started":1}
I20260924 21:16:11.984364 26600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:11.978637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55608 (local address 127.6.210.130:35753)
0924 21:16:11.979078 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:11.979082 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:11.979090 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:16:11.979124 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:11.979126 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:11.979141 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:11.979282 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:11.979485 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:11.980502 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:11.982823 (+ 2321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:11.983488 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:11.983998 (+ 510us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:11.984066 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:11.984095 (+ 29us) server_negotiation.cc:299] Negotiation successful
0924 21:16:11.984147 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":83,"threads_started":1}
I20260924 21:16:11.984365 26253 tablet_service.cc:1467] Processing CreateTablet for tablet 347289049cf5422982e8e891aebe8fc7 (DEFAULT_TABLE table=test-table [id=7955b2ef19bf470b8ea232e075a3aad9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:11.984771 26253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 347289049cf5422982e8e891aebe8fc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:11.984920 26387 tablet_service.cc:1467] Processing CreateTablet for tablet 347289049cf5422982e8e891aebe8fc7 (DEFAULT_TABLE table=test-table [id=7955b2ef19bf470b8ea232e075a3aad9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:11.985148 26387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 347289049cf5422982e8e891aebe8fc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:11.986946 26521 tablet_service.cc:1467] Processing CreateTablet for tablet 347289049cf5422982e8e891aebe8fc7 (DEFAULT_TABLE table=test-table [id=7955b2ef19bf470b8ea232e075a3aad9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:11.987295 26521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 347289049cf5422982e8e891aebe8fc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:11.987785 26604 tablet_bootstrap.cc:492] T 347289049cf5422982e8e891aebe8fc7 P b0ceaf06e9ff4aa4bdf3c974e9217b10: Bootstrap starting.
I20260924 21:16:11.987898 26605 tablet_bootstrap.cc:492] T 347289049cf5422982e8e891aebe8fc7 P 22a9294bc5e649ce90ee61cc677b3e67: Bootstrap starting.
I20260924 21:16:11.988482 26604 tablet_bootstrap.cc:654] T 347289049cf5422982e8e891aebe8fc7 P b0ceaf06e9ff4aa4bdf3c974e9217b10: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:11.988547 26605 tablet_bootstrap.cc:654] T 347289049cf5422982e8e891aebe8fc7 P 22a9294bc5e649ce90ee61cc677b3e67: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:11.989006 26604 log.cc:824] T 347289049cf5422982e8e891aebe8fc7 P b0ceaf06e9ff4aa4bdf3c974e9217b10: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:11.989128 26605 log.cc:824] T 347289049cf5422982e8e891aebe8fc7 P 22a9294bc5e649ce90ee61cc677b3e67: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:11.989815 26605 tablet_bootstrap.cc:492] T 347289049cf5422982e8e891aebe8fc7 P 22a9294bc5e649ce90ee61cc677b3e67: No bootstrap required, opened a new log
I20260924 21:16:11.989841 26604 tablet_bootstrap.cc:492] T 347289049cf5422982e8e891aebe8fc7 P b0ceaf06e9ff4aa4bdf3c974e9217b10: No bootstrap required, opened a new log
I20260924 21:16:11.989897 26604 ts_tablet_manager.cc:1397] T 347289049cf5422982e8e891aebe8fc7 P b0ceaf06e9ff4aa4bdf3c974e9217b10: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:16:11.989899 26605 ts_tablet_manager.cc:1397] T 347289049cf5422982e8e891aebe8fc7 P 22a9294bc5e649ce90ee61cc677b3e67: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:16:11.990640 26608 tablet_bootstrap.cc:492] T 347289049cf5422982e8e891aebe8fc7 P 480b86b3f96e42509b520dfb58576237: Bootstrap starting.
I20260924 21:16:11.991466 26608 tablet_bootstrap.cc:654] T 347289049cf5422982e8e891aebe8fc7 P 480b86b3f96e42509b520dfb58576237: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:11.991724 26604 raft_consensus.cc:348] T 347289049cf5422982e8e891aebe8fc7 P b0ceaf06e9ff4aa4bdf3c974e9217b10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a9294bc5e649ce90ee61cc677b3e67" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 35753 } } peers { permanent_uuid: "480b86b3f96e42509b520dfb58576237" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 34031 } } peers { permanent_uuid: "b0ceaf06e9ff4aa4bdf3c974e9217b10" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 34473 } }
I20260924 21:16:11.991879 26604 raft_consensus.cc:374] T 347289049cf5422982e8e891aebe8fc7 P b0ceaf06e9ff4aa4bdf3c974e9217b10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:11.991921 26604 raft_consensus.cc:729] T 347289049cf5422982e8e891aebe8fc7 P b0ceaf06e9ff4aa4bdf3c974e9217b10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0ceaf06e9ff4aa4bdf3c974e9217b10, State: Initialized, Role: FOLLOWER
I20260924 21:16:11.992012 26608 log.cc:824] T 347289049cf5422982e8e891aebe8fc7 P 480b86b3f96e42509b520dfb58576237: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:11.991917 26605 raft_consensus.cc:348] T 347289049cf5422982e8e891aebe8fc7 P 22a9294bc5e649ce90ee61cc677b3e67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a9294bc5e649ce90ee61cc677b3e67" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 35753 } } peers { permanent_uuid: "480b86b3f96e42509b520dfb58576237" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 34031 } } peers { permanent_uuid: "b0ceaf06e9ff4aa4bdf3c974e9217b10" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 34473 } }
I20260924 21:16:11.992082 26605 raft_consensus.cc:374] T 347289049cf5422982e8e891aebe8fc7 P 22a9294bc5e649ce90ee61cc677b3e67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:11.992035 26604 consensus_queue.cc:260] T 347289049cf5422982e8e891aebe8fc7 P b0ceaf06e9ff4aa4bdf3c974e9217b10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a9294bc5e649ce90ee61cc677b3e67" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 35753 } } peers { permanent_uuid: "480b86b3f96e42509b520dfb58576237" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 34031 } } peers { permanent_uuid: "b0ceaf06e9ff4aa4bdf3c974e9217b10" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 34473 } }
I20260924 21:16:11.992115 26605 raft_consensus.cc:729] T 347289049cf5422982e8e891aebe8fc7 P 22a9294bc5e649ce90ee61cc677b3e67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22a9294bc5e649ce90ee61cc677b3e67, State: Initialized, Role: FOLLOWER
I20260924 21:16:11.992256 26605 consensus_queue.cc:260] T 347289049cf5422982e8e891aebe8fc7 P 22a9294bc5e649ce90ee61cc677b3e67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a9294bc5e649ce90ee61cc677b3e67" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 35753 } } peers { permanent_uuid: "480b86b3f96e42509b520dfb58576237" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 34031 } } peers { permanent_uuid: "b0ceaf06e9ff4aa4bdf3c974e9217b10" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 34473 } }
I20260924 21:16:11.992340 26604 ts_tablet_manager.cc:1428] T 347289049cf5422982e8e891aebe8fc7 P b0ceaf06e9ff4aa4bdf3c974e9217b10: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 21:16:11.992379 26319 heartbeater.cc:503] Master 127.6.210.190:41439 was elected leader, sending a full tablet report...
I20260924 21:16:11.992534 26605 ts_tablet_manager.cc:1428] T 347289049cf5422982e8e891aebe8fc7 P 22a9294bc5e649ce90ee61cc677b3e67: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260924 21:16:11.992563 26453 heartbeater.cc:503] Master 127.6.210.190:41439 was elected leader, sending a full tablet report...
I20260924 21:16:11.992938 26608 tablet_bootstrap.cc:492] T 347289049cf5422982e8e891aebe8fc7 P 480b86b3f96e42509b520dfb58576237: No bootstrap required, opened a new log
I20260924 21:16:11.992995 26608 ts_tablet_manager.cc:1397] T 347289049cf5422982e8e891aebe8fc7 P 480b86b3f96e42509b520dfb58576237: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:16:11.994661 26608 raft_consensus.cc:348] T 347289049cf5422982e8e891aebe8fc7 P 480b86b3f96e42509b520dfb58576237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a9294bc5e649ce90ee61cc677b3e67" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 35753 } } peers { permanent_uuid: "480b86b3f96e42509b520dfb58576237" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 34031 } } peers { permanent_uuid: "b0ceaf06e9ff4aa4bdf3c974e9217b10" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 34473 } }
I20260924 21:16:11.994809 26608 raft_consensus.cc:374] T 347289049cf5422982e8e891aebe8fc7 P 480b86b3f96e42509b520dfb58576237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:11.994874 26608 raft_consensus.cc:729] T 347289049cf5422982e8e891aebe8fc7 P 480b86b3f96e42509b520dfb58576237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 480b86b3f96e42509b520dfb58576237, State: Initialized, Role: FOLLOWER
I20260924 21:16:11.994982 26608 consensus_queue.cc:260] T 347289049cf5422982e8e891aebe8fc7 P 480b86b3f96e42509b520dfb58576237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a9294bc5e649ce90ee61cc677b3e67" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 35753 } } peers { permanent_uuid: "480b86b3f96e42509b520dfb58576237" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 34031 } } peers { permanent_uuid: "b0ceaf06e9ff4aa4bdf3c974e9217b10" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 34473 } }
I20260924 21:16:11.995253 26608 ts_tablet_manager.cc:1428] T 347289049cf5422982e8e891aebe8fc7 P 480b86b3f96e42509b520dfb58576237: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 21:16:11.995290 26587 heartbeater.cc:503] Master 127.6.210.190:41439 was elected leader, sending a full tablet report...
W20260924 21:16:12.078747 26320 tablet.cc:2369] T 347289049cf5422982e8e891aebe8fc7 P b0ceaf06e9ff4aa4bdf3c974e9217b10: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:16:12.169507 26610 raft_consensus.cc:482] T 347289049cf5422982e8e891aebe8fc7 P b0ceaf06e9ff4aa4bdf3c974e9217b10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:16:12.169741 26610 raft_consensus.cc:504] T 347289049cf5422982e8e891aebe8fc7 P b0ceaf06e9ff4aa4bdf3c974e9217b10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a9294bc5e649ce90ee61cc677b3e67" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 35753 } } peers { permanent_uuid: "480b86b3f96e42509b520dfb58576237" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 34031 } } peers { permanent_uuid: "b0ceaf06e9ff4aa4bdf3c974e9217b10" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 34473 } }
I20260924 21:16:12.170153 26610 leader_election.cc:288] T 347289049cf5422982e8e891aebe8fc7 P b0ceaf06e9ff4aa4bdf3c974e9217b10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22a9294bc5e649ce90ee61cc677b3e67 (127.6.210.130:35753), 480b86b3f96e42509b520dfb58576237 (127.6.210.131:34031)
I20260924 21:16:12.174613 26603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:12.170708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:46219 (local address 127.6.210.131:34031)
0924 21:16:12.170764 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:12.170766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:12.170870 (+ 104us) server_negotiation.cc:407] Connection header received
0924 21:16:12.170897 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:12.170898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:12.170908 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:12.170968 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:12.171119 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.171943 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.173516 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.174227 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.174344 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:12.174353 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:12.174495 (+ 142us) server_negotiation.cc:299] Negotiation successful
0924 21:16:12.174539 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 21:16:12.174623 26614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:12.170625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:34031 (local address 127.6.210.129:46219)
0924 21:16:12.170736 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:12.170746 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:12.170835 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:12.170976 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:12.170977 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:12.170978 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:12.171042 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:12.171044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.172044 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.172045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:12.173396 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:12.173400 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.174354 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.174355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:12.174415 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:12.174418 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:16:12.174472 (+ 54us) client_negotiation.cc:240] Negotiation successful
0924 21:16:12.174514 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":76,"thread_start_us":47,"threads_started":1}
I20260924 21:16:12.174613 26600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:12.170485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:47711 (local address 127.6.210.130:35753)
0924 21:16:12.170567 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:12.170570 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:12.170689 (+ 119us) server_negotiation.cc:407] Connection header received
0924 21:16:12.170806 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:12.170807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:12.170819 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:12.170887 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:12.171115 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.171945 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.173509 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.174164 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.174292 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:12.174304 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:12.174495 (+ 191us) server_negotiation.cc:299] Negotiation successful
0924 21:16:12.174539 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260924 21:16:12.174625 26613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:12.170399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:35753 (local address 127.6.210.129:47711)
0924 21:16:12.170581 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:12.170594 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:12.170698 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:12.170911 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:12.170912 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:12.170918 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:12.171010 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:12.171012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.172044 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.172045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:12.173396 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:12.173400 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.174301 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.174302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:12.174394 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:12.174399 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 21:16:12.174463 (+ 64us) client_negotiation.cc:240] Negotiation successful
0924 21:16:12.174513 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":50,"thread_start_us":94,"threads_started":1}
I20260924 21:16:12.175083 26541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "347289049cf5422982e8e891aebe8fc7" candidate_uuid: "b0ceaf06e9ff4aa4bdf3c974e9217b10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "480b86b3f96e42509b520dfb58576237" is_pre_election: true
I20260924 21:16:12.175084 26407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "347289049cf5422982e8e891aebe8fc7" candidate_uuid: "b0ceaf06e9ff4aa4bdf3c974e9217b10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22a9294bc5e649ce90ee61cc677b3e67" is_pre_election: true
I20260924 21:16:12.175252 26541 raft_consensus.cc:2454] T 347289049cf5422982e8e891aebe8fc7 P 480b86b3f96e42509b520dfb58576237 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0ceaf06e9ff4aa4bdf3c974e9217b10 in term 0.
I20260924 21:16:12.175344 26407 raft_consensus.cc:2454] T 347289049cf5422982e8e891aebe8fc7 P 22a9294bc5e649ce90ee61cc677b3e67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0ceaf06e9ff4aa4bdf3c974e9217b10 in term 0.
I20260924 21:16:12.175503 26206 leader_election.cc:302] T 347289049cf5422982e8e891aebe8fc7 P b0ceaf06e9ff4aa4bdf3c974e9217b10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 480b86b3f96e42509b520dfb58576237, b0ceaf06e9ff4aa4bdf3c974e9217b10; no voters:
I20260924 21:16:12.175619 26610 raft_consensus.cc:2781] T 347289049cf5422982e8e891aebe8fc7 P b0ceaf06e9ff4aa4bdf3c974e9217b10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:16:12.175717 26610 raft_consensus.cc:482] T 347289049cf5422982e8e891aebe8fc7 P b0ceaf06e9ff4aa4bdf3c974e9217b10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:16:12.175755 26610 raft_consensus.cc:3037] T 347289049cf5422982e8e891aebe8fc7 P b0ceaf06e9ff4aa4bdf3c974e9217b10 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:12.176527 26610 raft_consensus.cc:504] T 347289049cf5422982e8e891aebe8fc7 P b0ceaf06e9ff4aa4bdf3c974e9217b10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a9294bc5e649ce90ee61cc677b3e67" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 35753 } } peers { permanent_uuid: "480b86b3f96e42509b520dfb58576237" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 34031 } } peers { permanent_uuid: "b0ceaf06e9ff4aa4bdf3c974e9217b10" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 34473 } }
I20260924 21:16:12.176685 26610 leader_election.cc:288] T 347289049cf5422982e8e891aebe8fc7 P b0ceaf06e9ff4aa4bdf3c974e9217b10 [CANDIDATE]: Term 1 election: Requested vote from peers 22a9294bc5e649ce90ee61cc677b3e67 (127.6.210.130:35753), 480b86b3f96e42509b520dfb58576237 (127.6.210.131:34031)
I20260924 21:16:12.176836 26407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "347289049cf5422982e8e891aebe8fc7" candidate_uuid: "b0ceaf06e9ff4aa4bdf3c974e9217b10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22a9294bc5e649ce90ee61cc677b3e67"
I20260924 21:16:12.176877 26541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "347289049cf5422982e8e891aebe8fc7" candidate_uuid: "b0ceaf06e9ff4aa4bdf3c974e9217b10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "480b86b3f96e42509b520dfb58576237"
I20260924 21:16:12.176940 26407 raft_consensus.cc:3037] T 347289049cf5422982e8e891aebe8fc7 P 22a9294bc5e649ce90ee61cc677b3e67 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:12.176966 26541 raft_consensus.cc:3037] T 347289049cf5422982e8e891aebe8fc7 P 480b86b3f96e42509b520dfb58576237 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:12.177671 26541 raft_consensus.cc:2454] T 347289049cf5422982e8e891aebe8fc7 P 480b86b3f96e42509b520dfb58576237 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0ceaf06e9ff4aa4bdf3c974e9217b10 in term 1.
I20260924 21:16:12.177681 26407 raft_consensus.cc:2454] T 347289049cf5422982e8e891aebe8fc7 P 22a9294bc5e649ce90ee61cc677b3e67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0ceaf06e9ff4aa4bdf3c974e9217b10 in term 1.
I20260924 21:16:12.177840 26206 leader_election.cc:302] T 347289049cf5422982e8e891aebe8fc7 P b0ceaf06e9ff4aa4bdf3c974e9217b10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22a9294bc5e649ce90ee61cc677b3e67, b0ceaf06e9ff4aa4bdf3c974e9217b10; no voters:
I20260924 21:16:12.177925 26610 raft_consensus.cc:2781] T 347289049cf5422982e8e891aebe8fc7 P b0ceaf06e9ff4aa4bdf3c974e9217b10 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:12.178061 26610 raft_consensus.cc:686] T 347289049cf5422982e8e891aebe8fc7 P b0ceaf06e9ff4aa4bdf3c974e9217b10 [term 1 LEADER]: Becoming Leader. State: Replica: b0ceaf06e9ff4aa4bdf3c974e9217b10, State: Running, Role: LEADER
I20260924 21:16:12.178156 26610 consensus_queue.cc:237] T 347289049cf5422982e8e891aebe8fc7 P b0ceaf06e9ff4aa4bdf3c974e9217b10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a9294bc5e649ce90ee61cc677b3e67" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 35753 } } peers { permanent_uuid: "480b86b3f96e42509b520dfb58576237" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 34031 } } peers { permanent_uuid: "b0ceaf06e9ff4aa4bdf3c974e9217b10" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 34473 } }
I20260924 21:16:12.179096 26127 catalog_manager.cc:5495] T 347289049cf5422982e8e891aebe8fc7 P b0ceaf06e9ff4aa4bdf3c974e9217b10 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0ceaf06e9ff4aa4bdf3c974e9217b10 (127.6.210.129). New cstate: current_term: 1 leader_uuid: "b0ceaf06e9ff4aa4bdf3c974e9217b10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22a9294bc5e649ce90ee61cc677b3e67" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 35753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "480b86b3f96e42509b520dfb58576237" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 34031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0ceaf06e9ff4aa4bdf3c974e9217b10" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 34473 } health_report { overall_health: HEALTHY } } }
W20260924 21:16:12.198133 26588 tablet.cc:2369] T 347289049cf5422982e8e891aebe8fc7 P 480b86b3f96e42509b520dfb58576237: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:16:12.209983 26601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:12.207575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57708 (local address 127.6.210.129:34473)
0924 21:16:12.207640 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:12.207642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:12.207660 (+ 18us) server_negotiation.cc:407] Connection header received
0924 21:16:12.207745 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:12.207746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:12.207765 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:12.207821 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:16:12.207945 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.208432 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.209204 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.209398 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.209461 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:12.209609 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:16:12.209728 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:16:12.209791 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:12.209832 (+ 41us) server_negotiation.cc:299] Negotiation successful
0924 21:16:12.209852 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 21:16:12.211858 26541 raft_consensus.cc:1261] T 347289049cf5422982e8e891aebe8fc7 P 480b86b3f96e42509b520dfb58576237 [term 1 FOLLOWER]: Refusing update from remote peer b0ceaf06e9ff4aa4bdf3c974e9217b10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:16:12.211931 26407 raft_consensus.cc:1261] T 347289049cf5422982e8e891aebe8fc7 P 22a9294bc5e649ce90ee61cc677b3e67 [term 1 FOLLOWER]: Refusing update from remote peer b0ceaf06e9ff4aa4bdf3c974e9217b10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:16:12.212189 26610 consensus_queue.cc:1035] T 347289049cf5422982e8e891aebe8fc7 P b0ceaf06e9ff4aa4bdf3c974e9217b10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "480b86b3f96e42509b520dfb58576237" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 34031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:16:12.212320 26615 consensus_queue.cc:1035] T 347289049cf5422982e8e891aebe8fc7 P b0ceaf06e9ff4aa4bdf3c974e9217b10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22a9294bc5e649ce90ee61cc677b3e67" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 35753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:16:12.214617 26618 mvcc.cc:205] Tried to move back new op lower bound from 7333005607777153024 to 7333005607642406912. Current Snapshot: MvccSnapshot[applied={T|T < 7333005607777153024}]
I20260924 21:16:12.215109 26620 mvcc.cc:205] Tried to move back new op lower bound from 7333005607777153024 to 7333005607642406912. Current Snapshot: MvccSnapshot[applied={T|T < 7333005607777153024}]
I20260924 21:16:12.217250 26128 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41194:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:16:12.217373 26128 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41194:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:16:12.218084 26128 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 48b89b8ccec04af2886e87106460f1cd: Sending DeleteTablet for 3 replicas of tablet 347289049cf5422982e8e891aebe8fc7
I20260924 21:16:12.218444 26253 tablet_service.cc:1514] Processing DeleteTablet for tablet 347289049cf5422982e8e891aebe8fc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:12 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:57694
I20260924 21:16:12.218487 26521 tablet_service.cc:1514] Processing DeleteTablet for tablet 347289049cf5422982e8e891aebe8fc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:12 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:54780
I20260924 21:16:12.218544 26387 tablet_service.cc:1514] Processing DeleteTablet for tablet 347289049cf5422982e8e891aebe8fc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:12 UTC) from {username='kudu', principal='kudu/127.6.210.190@KRBTEST.COM'} at 127.0.0.1:55608
I20260924 21:16:12.218818 26627 tablet_replica.cc:330] T 347289049cf5422982e8e891aebe8fc7 P b0ceaf06e9ff4aa4bdf3c974e9217b10: stopping tablet replica
I20260924 21:16:12.218873 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 26190
I20260924 21:16:12.218955 26628 tablet_replica.cc:330] T 347289049cf5422982e8e891aebe8fc7 P 480b86b3f96e42509b520dfb58576237: stopping tablet replica
I20260924 21:16:12.219054 26628 raft_consensus.cc:2229] T 347289049cf5422982e8e891aebe8fc7 P 480b86b3f96e42509b520dfb58576237 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:16:12.219151 26628 raft_consensus.cc:2258] T 347289049cf5422982e8e891aebe8fc7 P 480b86b3f96e42509b520dfb58576237 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:16:12.219290 26627 raft_consensus.cc:2229] T 347289049cf5422982e8e891aebe8fc7 P b0ceaf06e9ff4aa4bdf3c974e9217b10 [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:16:12.219631 26629 tablet_replica.cc:330] T 347289049cf5422982e8e891aebe8fc7 P 22a9294bc5e649ce90ee61cc677b3e67: stopping tablet replica
I20260924 21:16:12.219717 26629 raft_consensus.cc:2229] T 347289049cf5422982e8e891aebe8fc7 P 22a9294bc5e649ce90ee61cc677b3e67 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:16:12.219738 26628 ts_tablet_manager.cc:1905] T 347289049cf5422982e8e891aebe8fc7 P 480b86b3f96e42509b520dfb58576237: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:16:12.219838 26629 raft_consensus.cc:2258] T 347289049cf5422982e8e891aebe8fc7 P 22a9294bc5e649ce90ee61cc677b3e67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:16:12.220402 26629 ts_tablet_manager.cc:1905] T 347289049cf5422982e8e891aebe8fc7 P 22a9294bc5e649ce90ee61cc677b3e67: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:16:12.221863 26628 ts_tablet_manager.cc:1918] T 347289049cf5422982e8e891aebe8fc7 P 480b86b3f96e42509b520dfb58576237: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:16:12.221939 26628 log.cc:1196] T 347289049cf5422982e8e891aebe8fc7 P 480b86b3f96e42509b520dfb58576237: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-2/wal/wals/347289049cf5422982e8e891aebe8fc7
I20260924 21:16:12.222296 26629 ts_tablet_manager.cc:1918] T 347289049cf5422982e8e891aebe8fc7 P 22a9294bc5e649ce90ee61cc677b3e67: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:16:12.222321 26628 ts_tablet_manager.cc:1939] T 347289049cf5422982e8e891aebe8fc7 P 480b86b3f96e42509b520dfb58576237: Deleting consensus metadata
I20260924 21:16:12.222371 26629 log.cc:1196] T 347289049cf5422982e8e891aebe8fc7 P 22a9294bc5e649ce90ee61cc677b3e67: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284333163253-6986-0/minicluster-data/ts-1/wal/wals/347289049cf5422982e8e891aebe8fc7
I20260924 21:16:12.222610 26629 ts_tablet_manager.cc:1939] T 347289049cf5422982e8e891aebe8fc7 P 22a9294bc5e649ce90ee61cc677b3e67: Deleting consensus metadata
I20260924 21:16:12.222781 26112 catalog_manager.cc:4841] TS 480b86b3f96e42509b520dfb58576237 (127.6.210.131:34031): tablet 347289049cf5422982e8e891aebe8fc7 (table test-table [id=7955b2ef19bf470b8ea232e075a3aad9]) successfully deleted
I20260924 21:16:12.223021 26112 catalog_manager.cc:4841] TS 22a9294bc5e649ce90ee61cc677b3e67 (127.6.210.130:35753): tablet 347289049cf5422982e8e891aebe8fc7 (table test-table [id=7955b2ef19bf470b8ea232e075a3aad9]) successfully deleted
W20260924 21:16:12.224943 26114 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.210.129:34473 (error 108)
I20260924 21:16:12.225430 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 26324
I20260924 21:16:12.225543 26602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:12.225371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:34473 (local address 127.0.0.1:57710)
0924 21:16:12.225447 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:12.225506 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.129:34473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260924 21:16:12.226177 26114 catalog_manager.cc:4584] TS b0ceaf06e9ff4aa4bdf3c974e9217b10 (127.6.210.129:34473): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 347289049cf5422982e8e891aebe8fc7: Network error: Client connection negotiation failed: client connection to 127.6.210.129:34473: connect: Connection refused (error 111)
I20260924 21:16:12.230038 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 26458
I20260924 21:16:12.234673 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 26096
2026-09-24T21:16:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2174 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-24T21:16:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:16:12Z Disabled control of system clock
I20260924 21:16:12.262236 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:42911
--webserver_interface=127.6.210.190
--webserver_port=0
--builtin_ntp_servers=127.6.210.148:42431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:42911
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 21:16:12.341951 26634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:12.342177 26634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:12.342197 26634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:12.343428 26634 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:12.343473 26634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:12.343487 26634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:12.343499 26634 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:12.343538 26634 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:12.344918 26634 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:42431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:42911
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:42911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:12.345171 26634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:12.345498 26634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:12.350481 26639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:12.350489 26640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:12.350539 26634 server_base.cc:1034] running on GCE node
W20260924 21:16:12.350474 26642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:12.350925 26634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:12.351290 26634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:12.352466 26634 hybrid_clock.cc:648] HybridClock initialized: now 1790284572352450 us; error 38 us; skew 500 ppm
I20260924 21:16:12.353672 26634 webserver.cc:460] Webserver started at http://127.6.210.190:46359/ using document root <none> and password file <none>
I20260924 21:16:12.353929 26634 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:12.353998 26634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:12.354130 26634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:12.355296 26634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "bdd47bc119614d2e8359bbf575c9295c"
format_stamp: "Formatted at 2026-09-24 21:16:12 on dist-test-slave-q276"
server_key: "36fccad0235cfcf94f5221f9ff3ebacf"
server_key_iv: "a37814d9c5a9c23b1c3c2eabefd9fb80"
server_key_version: "encryptionkey@0"
I20260924 21:16:12.355672 26634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "bdd47bc119614d2e8359bbf575c9295c"
format_stamp: "Formatted at 2026-09-24 21:16:12 on dist-test-slave-q276"
server_key: "36fccad0235cfcf94f5221f9ff3ebacf"
server_key_iv: "a37814d9c5a9c23b1c3c2eabefd9fb80"
server_key_version: "encryptionkey@0"
I20260924 21:16:12.357019 26634 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:16:12.357762 26648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:12.357942 26634 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:16:12.358042 26634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "bdd47bc119614d2e8359bbf575c9295c"
format_stamp: "Formatted at 2026-09-24 21:16:12 on dist-test-slave-q276"
server_key: "36fccad0235cfcf94f5221f9ff3ebacf"
server_key_iv: "a37814d9c5a9c23b1c3c2eabefd9fb80"
server_key_version: "encryptionkey@0"
I20260924 21:16:12.358130 26634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:12.370239 26634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:12.370653 26634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:12.370821 26634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:12.374747 26634 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:42911
I20260924 21:16:12.374782 26700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:42911 every 8 connection(s)
I20260924 21:16:12.375121 26634 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:16:12.376051 26701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:12.376482 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 26634
I20260924 21:16:12.376583 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:16:12.376783 6986 external_mini_cluster.cc:1428] Setting key 1cd6e0fa0976d6d365780bd3d51490e5
I20260924 21:16:12.378996 26701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdd47bc119614d2e8359bbf575c9295c: Bootstrap starting.
I20260924 21:16:12.379683 26701 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bdd47bc119614d2e8359bbf575c9295c: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:12.379983 26701 log.cc:824] T 00000000000000000000000000000000 P bdd47bc119614d2e8359bbf575c9295c: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:12.380690 26701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdd47bc119614d2e8359bbf575c9295c: No bootstrap required, opened a new log
I20260924 21:16:12.382498 26704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:12.377320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35188 (local address 127.6.210.190:42911)
0924 21:16:12.377519 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:12.377526 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:12.377550 (+ 24us) server_negotiation.cc:407] Connection header received
0924 21:16:12.377967 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:12.377973 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:12.378032 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:12.378146 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:12.378408 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.379339 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.380470 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.380922 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.381012 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:12.381701 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:12.381705 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:12.381706 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:12.381918 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:12.381955 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:12.381967 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:12.382037 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:12.382147 (+ 110us) server_negotiation.cc:299] Negotiation successful
0924 21:16:12.382261 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260924 21:16:12.383325 26701 raft_consensus.cc:348] T 00000000000000000000000000000000 P bdd47bc119614d2e8359bbf575c9295c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd47bc119614d2e8359bbf575c9295c" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 42911 } }
I20260924 21:16:12.383490 26701 raft_consensus.cc:374] T 00000000000000000000000000000000 P bdd47bc119614d2e8359bbf575c9295c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:12.383525 26701 raft_consensus.cc:729] T 00000000000000000000000000000000 P bdd47bc119614d2e8359bbf575c9295c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdd47bc119614d2e8359bbf575c9295c, State: Initialized, Role: FOLLOWER
I20260924 21:16:12.383673 26701 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdd47bc119614d2e8359bbf575c9295c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd47bc119614d2e8359bbf575c9295c" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 42911 } }
I20260924 21:16:12.383766 26701 raft_consensus.cc:388] T 00000000000000000000000000000000 P bdd47bc119614d2e8359bbf575c9295c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:16:12.383803 26701 raft_consensus.cc:482] T 00000000000000000000000000000000 P bdd47bc119614d2e8359bbf575c9295c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:16:12.383876 26701 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bdd47bc119614d2e8359bbf575c9295c [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:12.384662 26701 raft_consensus.cc:504] T 00000000000000000000000000000000 P bdd47bc119614d2e8359bbf575c9295c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd47bc119614d2e8359bbf575c9295c" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 42911 } }
I20260924 21:16:12.384793 26701 leader_election.cc:302] T 00000000000000000000000000000000 P bdd47bc119614d2e8359bbf575c9295c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bdd47bc119614d2e8359bbf575c9295c; no voters:
I20260924 21:16:12.384960 26701 leader_election.cc:288] T 00000000000000000000000000000000 P bdd47bc119614d2e8359bbf575c9295c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:16:12.385037 26706 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bdd47bc119614d2e8359bbf575c9295c [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:12.385236 26706 raft_consensus.cc:686] T 00000000000000000000000000000000 P bdd47bc119614d2e8359bbf575c9295c [term 1 LEADER]: Becoming Leader. State: Replica: bdd47bc119614d2e8359bbf575c9295c, State: Running, Role: LEADER
I20260924 21:16:12.385241 26701 sys_catalog.cc:564] T 00000000000000000000000000000000 P bdd47bc119614d2e8359bbf575c9295c [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:12.385388 26706 consensus_queue.cc:237] T 00000000000000000000000000000000 P bdd47bc119614d2e8359bbf575c9295c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd47bc119614d2e8359bbf575c9295c" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 42911 } }
I20260924 21:16:12.385839 26706 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdd47bc119614d2e8359bbf575c9295c [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdd47bc119614d2e8359bbf575c9295c. Latest consensus state: current_term: 1 leader_uuid: "bdd47bc119614d2e8359bbf575c9295c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd47bc119614d2e8359bbf575c9295c" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 42911 } } }
I20260924 21:16:12.385946 26706 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdd47bc119614d2e8359bbf575c9295c [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:12.386497 26714 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:16:12.386791 26707 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdd47bc119614d2e8359bbf575c9295c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bdd47bc119614d2e8359bbf575c9295c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd47bc119614d2e8359bbf575c9295c" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 42911 } } }
I20260924 21:16:12.386883 26707 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdd47bc119614d2e8359bbf575c9295c [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:12.387324 26714 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:16:12.389320 26714 catalog_manager.cc:1347] Generated new cluster ID: dd5d0b3287374ea6807e6f7e5c36f511
I20260924 21:16:12.389386 26714 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:16:12.397341 26714 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:16:12.397969 26714 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:16:12.402992 26714 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bdd47bc119614d2e8359bbf575c9295c: Generated new TSK 0
I20260924 21:16:12.403206 26714 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:16:12.412540 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.129:0
--local_ip_for_outbound_sockets=127.6.210.129
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:42911
--builtin_ntp_servers=127.6.210.148:42431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 21:16:12.494918 26725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:12.495158 26725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:12.495194 26725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:12.496342 26725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:12.496405 26725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:12.496446 26725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.129
I20260924 21:16:12.497809 26725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:42431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.210.129
--webserver_port=0
--tserver_master_addrs=127.6.210.190:42911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.129
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:12.498075 26725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:12.498442 26725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:12.503546 26733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:12.503556 26730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:12.503688 26725 server_base.cc:1034] running on GCE node
W20260924 21:16:12.503813 26731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:12.504073 26725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:12.504343 26725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:12.505528 26725 hybrid_clock.cc:648] HybridClock initialized: now 1790284572505494 us; error 51 us; skew 500 ppm
I20260924 21:16:12.507031 26725 webserver.cc:460] Webserver started at http://127.6.210.129:45515/ using document root <none> and password file <none>
I20260924 21:16:12.507362 26725 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:12.507452 26725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:12.507594 26725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:12.509121 26725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-0/data/instance:
uuid: "32964048b2fb4b4792a6954e5d71b997"
format_stamp: "Formatted at 2026-09-24 21:16:12 on dist-test-slave-q276"
server_key: "9e27e46217ac4f554b99bc9e34fe3187"
server_key_iv: "393f41033ac3cea99a2138a14eb0d2f9"
server_key_version: "encryptionkey@0"
I20260924 21:16:12.509521 26725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance:
uuid: "32964048b2fb4b4792a6954e5d71b997"
format_stamp: "Formatted at 2026-09-24 21:16:12 on dist-test-slave-q276"
server_key: "9e27e46217ac4f554b99bc9e34fe3187"
server_key_iv: "393f41033ac3cea99a2138a14eb0d2f9"
server_key_version: "encryptionkey@0"
I20260924 21:16:12.511157 26725 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260924 21:16:12.511996 26739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:12.512192 26725 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:12.512284 26725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-0/wal
uuid: "32964048b2fb4b4792a6954e5d71b997"
format_stamp: "Formatted at 2026-09-24 21:16:12 on dist-test-slave-q276"
server_key: "9e27e46217ac4f554b99bc9e34fe3187"
server_key_iv: "393f41033ac3cea99a2138a14eb0d2f9"
server_key_version: "encryptionkey@0"
I20260924 21:16:12.512352 26725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:12.536175 26725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:12.536466 26725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:12.536576 26725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:12.536778 26725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:16:12.537140 26725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:16:12.537177 26725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:12.537201 26725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:16:12.537292 26725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:12.543148 26725 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.129:34241
I20260924 21:16:12.543188 26852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.129:34241 every 8 connection(s)
I20260924 21:16:12.543741 26725 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-0/data/info.pb
I20260924 21:16:12.547914 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 26725
I20260924 21:16:12.548041 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-0/wal/instance
I20260924 21:16:12.548272 6986 external_mini_cluster.cc:1428] Setting key b40dce483d86657f61b396b41ed41bad
I20260924 21:16:12.549340 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.130:0
--local_ip_for_outbound_sockets=127.6.210.130
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:42911
--builtin_ntp_servers=127.6.210.148:42431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260924 21:16:12.549909 26704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:12.544631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.129:44821 (local address 127.6.210.190:42911)
0924 21:16:12.544713 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:12.544715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:12.544965 (+ 250us) server_negotiation.cc:407] Connection header received
0924 21:16:12.545574 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:12.545576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:12.545589 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:12.545648 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:12.546397 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.547046 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.548039 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.548289 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.548378 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:12.549532 (+ 1154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:12.549534 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:12.549534 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:12.549539 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:12.549557 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:12.549561 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:12.549635 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:12.549747 (+ 112us) server_negotiation.cc:299] Negotiation successful
0924 21:16:12.549785 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260924 21:16:12.550040 26855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:12.544618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:42911 (local address 127.6.210.129:44821)
0924 21:16:12.544828 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:12.544865 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:12.545439 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:12.545729 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:12.545732 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:12.545741 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:12.546246 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:12.546251 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.547146 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.547147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:12.547942 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:12.547944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.548438 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.548439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:12.548577 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:12.549336 (+ 759us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:16:12.549336 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:12.549369 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:16:12.549423 (+ 54us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:12.549642 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:12.549647 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:12.549651 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:16:12.549730 (+ 79us) client_negotiation.cc:240] Negotiation successful
0924 21:16:12.549823 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":96,"threads_started":1}
I20260924 21:16:12.550571 26853 heartbeater.cc:347] Connected to a master server at 127.6.210.190:42911
I20260924 21:16:12.550724 26853 heartbeater.cc:464] Registering TS with master...
I20260924 21:16:12.550987 26853 heartbeater.cc:511] Master 127.6.210.190:42911 requested a full tablet report, sending...
I20260924 21:16:12.551625 26665 ts_manager.cc:194] Registered new tserver with Master: 32964048b2fb4b4792a6954e5d71b997 (127.6.210.129:34241)
I20260924 21:16:12.552721 26665 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.210.129:44821
W20260924 21:16:12.632860 26856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:12.633060 26856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:12.633081 26856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:12.634141 26856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:12.634172 26856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:12.634196 26856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.130
I20260924 21:16:12.635596 26856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:42431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.210.130
--webserver_port=0
--tserver_master_addrs=127.6.210.190:42911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.130
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:12.635787 26856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:12.636024 26856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:12.640981 26862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:12.640978 26864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:12.640996 26861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:12.641254 26856 server_base.cc:1034] running on GCE node
I20260924 21:16:12.641587 26856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:12.641916 26856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:12.643163 26856 hybrid_clock.cc:648] HybridClock initialized: now 1790284572643134 us; error 41 us; skew 500 ppm
I20260924 21:16:12.644532 26856 webserver.cc:460] Webserver started at http://127.6.210.130:46677/ using document root <none> and password file <none>
I20260924 21:16:12.644869 26856 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:12.644953 26856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:12.645128 26856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:12.646399 26856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-1/data/instance:
uuid: "82e0cf764edb439aa9a65e7683ea3ae4"
format_stamp: "Formatted at 2026-09-24 21:16:12 on dist-test-slave-q276"
server_key: "eb6723834658e3f7cbc97098cd3c24c2"
server_key_iv: "b98155cc4dff750f04d785c7db8f61bd"
server_key_version: "encryptionkey@0"
I20260924 21:16:12.646827 26856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance:
uuid: "82e0cf764edb439aa9a65e7683ea3ae4"
format_stamp: "Formatted at 2026-09-24 21:16:12 on dist-test-slave-q276"
server_key: "eb6723834658e3f7cbc97098cd3c24c2"
server_key_iv: "b98155cc4dff750f04d785c7db8f61bd"
server_key_version: "encryptionkey@0"
I20260924 21:16:12.648355 26856 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:16:12.649253 26870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:12.649441 26856 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:12.649555 26856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-1/wal
uuid: "82e0cf764edb439aa9a65e7683ea3ae4"
format_stamp: "Formatted at 2026-09-24 21:16:12 on dist-test-slave-q276"
server_key: "eb6723834658e3f7cbc97098cd3c24c2"
server_key_iv: "b98155cc4dff750f04d785c7db8f61bd"
server_key_version: "encryptionkey@0"
I20260924 21:16:12.649654 26856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:12.667798 26856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:12.668176 26856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:12.668339 26856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:12.668627 26856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:16:12.668972 26856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:16:12.669028 26856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:12.669083 26856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:16:12.669116 26856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:12.674923 26856 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.130:38993
I20260924 21:16:12.674962 26983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.130:38993 every 8 connection(s)
I20260924 21:16:12.675348 26856 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-1/data/info.pb
I20260924 21:16:12.680699 26704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:12.676094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:48647 (local address 127.6.210.190:42911)
0924 21:16:12.676180 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:12.676183 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:12.676416 (+ 233us) server_negotiation.cc:407] Connection header received
0924 21:16:12.676872 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:12.676873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:12.676885 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:12.676949 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:12.677592 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.678137 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.679231 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.679428 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.679490 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:12.680405 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:12.680407 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:12.680407 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:12.680411 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:12.680427 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:12.680430 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:12.680487 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:12.680610 (+ 123us) server_negotiation.cc:299] Negotiation successful
0924 21:16:12.680639 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260924 21:16:12.680948 26986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:12.676058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:42911 (local address 127.6.210.130:48647)
0924 21:16:12.676267 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:12.676303 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:12.676778 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:12.677002 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:12.677005 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:12.677013 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:12.677506 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:12.677510 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.678321 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.678322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:12.679122 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:12.679124 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.679508 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.679509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:12.679616 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:12.680219 (+ 603us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:16:12.680220 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:12.680254 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:16:12.680291 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:12.680519 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:12.680525 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:12.680528 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:16:12.680603 (+ 75us) client_negotiation.cc:240] Negotiation successful
0924 21:16:12.680703 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":98,"threads_started":1}
I20260924 21:16:12.681495 26984 heartbeater.cc:347] Connected to a master server at 127.6.210.190:42911
I20260924 21:16:12.681584 26984 heartbeater.cc:464] Registering TS with master...
I20260924 21:16:12.681730 26984 heartbeater.cc:511] Master 127.6.210.190:42911 requested a full tablet report, sending...
I20260924 21:16:12.682098 26665 ts_manager.cc:194] Registered new tserver with Master: 82e0cf764edb439aa9a65e7683ea3ae4 (127.6.210.130:38993)
I20260924 21:16:12.682526 26665 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.210.130:48647
I20260924 21:16:12.685423 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 26856
I20260924 21:16:12.685503 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-1/wal/instance
I20260924 21:16:12.685693 6986 external_mini_cluster.cc:1428] Setting key c14d09a96c72c9dde1e35ab2e7160ee8
I20260924 21:16:12.686653 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.210.131:0
--local_ip_for_outbound_sockets=127.6.210.131
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:42911
--builtin_ntp_servers=127.6.210.148:42431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 21:16:12.769680 26987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:12.769932 26987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:12.769955 26987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:12.771261 26987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:12.771302 26987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:12.771327 26987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.210.131
I20260924 21:16:12.772653 26987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:42431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.210.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.210.131
--webserver_port=0
--tserver_master_addrs=127.6.210.190:42911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.210.131
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:12.772892 26987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:12.773180 26987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:12.778086 26992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:12.778102 26993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:12.778226 26987 server_base.cc:1034] running on GCE node
W20260924 21:16:12.778102 26995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:12.778618 26987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:12.778828 26987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:12.779965 26987 hybrid_clock.cc:648] HybridClock initialized: now 1790284572779949 us; error 32 us; skew 500 ppm
I20260924 21:16:12.781062 26987 webserver.cc:460] Webserver started at http://127.6.210.131:45561/ using document root <none> and password file <none>
I20260924 21:16:12.781286 26987 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:12.781329 26987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:12.781415 26987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:12.782483 26987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-2/data/instance:
uuid: "3b1ee242da02445485cfc9be19fc625e"
format_stamp: "Formatted at 2026-09-24 21:16:12 on dist-test-slave-q276"
server_key: "b4068e65cea51d3046a8cff7fc3bb699"
server_key_iv: "1640851217539bb77f91c462b6d8f8f1"
server_key_version: "encryptionkey@0"
I20260924 21:16:12.782773 26987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance:
uuid: "3b1ee242da02445485cfc9be19fc625e"
format_stamp: "Formatted at 2026-09-24 21:16:12 on dist-test-slave-q276"
server_key: "b4068e65cea51d3046a8cff7fc3bb699"
server_key_iv: "1640851217539bb77f91c462b6d8f8f1"
server_key_version: "encryptionkey@0"
I20260924 21:16:12.784003 26987 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:12.784685 27001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:12.784868 26987 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:16:12.784970 26987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-2/wal
uuid: "3b1ee242da02445485cfc9be19fc625e"
format_stamp: "Formatted at 2026-09-24 21:16:12 on dist-test-slave-q276"
server_key: "b4068e65cea51d3046a8cff7fc3bb699"
server_key_iv: "1640851217539bb77f91c462b6d8f8f1"
server_key_version: "encryptionkey@0"
I20260924 21:16:12.785056 26987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:12.804059 26987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:12.804422 26987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:12.804584 26987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:12.804832 26987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:16:12.805179 26987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:16:12.805243 26987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:12.805291 26987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:16:12.805315 26987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:12.811547 26987 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.131:34851
I20260924 21:16:12.811597 27114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.131:34851 every 8 connection(s)
I20260924 21:16:12.811961 26987 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-2/data/info.pb
I20260924 21:16:12.817550 26704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:12.812873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.131:39381 (local address 127.6.210.190:42911)
0924 21:16:12.812969 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:12.812972 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:12.813181 (+ 209us) server_negotiation.cc:407] Connection header received
0924 21:16:12.813690 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:12.813691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:12.813703 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:12.813764 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:12.814441 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.814997 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.816047 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.816305 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.816377 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:12.817264 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:12.817272 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:12.817272 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:12.817277 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:12.817295 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:12.817300 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:12.817358 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:12.817448 (+ 90us) server_negotiation.cc:299] Negotiation successful
0924 21:16:12.817474 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260924 21:16:12.817727 27117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:12.812839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:42911 (local address 127.6.210.131:39381)
0924 21:16:12.813040 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:12.813077 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:12.813587 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:12.813818 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:12.813822 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:12.813830 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:12.814368 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:12.814373 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.815081 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.815082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:12.815966 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:12.815969 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.816392 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.816393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:12.816501 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:12.817104 (+ 603us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:16:12.817105 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:12.817133 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:16:12.817157 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:12.817366 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:12.817370 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:12.817373 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:16:12.817426 (+ 53us) client_negotiation.cc:240] Negotiation successful
0924 21:16:12.817507 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":119,"threads_started":1}
I20260924 21:16:12.818323 27115 heartbeater.cc:347] Connected to a master server at 127.6.210.190:42911
I20260924 21:16:12.818419 27115 heartbeater.cc:464] Registering TS with master...
I20260924 21:16:12.818621 27115 heartbeater.cc:511] Master 127.6.210.190:42911 requested a full tablet report, sending...
I20260924 21:16:12.819072 26665 ts_manager.cc:194] Registered new tserver with Master: 3b1ee242da02445485cfc9be19fc625e (127.6.210.131:34851)
I20260924 21:16:12.819442 26665 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.210.131:39381
I20260924 21:16:12.821517 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 26987
I20260924 21:16:12.821671 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-2/wal/instance
I20260924 21:16:12.821877 6986 external_mini_cluster.cc:1428] Setting key 9e2ca44fe48f371a6c82e5ddd6119cb3
I20260924 21:16:12.822897 6986 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:16:12.827199 26704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:12.824398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35192 (local address 127.6.210.190:42911)
0924 21:16:12.824476 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:12.824478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:12.824528 (+ 50us) server_negotiation.cc:407] Connection header received
0924 21:16:12.824582 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:12.824583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:12.824591 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:12.824639 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:12.824780 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.825255 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.826001 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.826169 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.826216 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:12.826969 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:12.826970 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:12.826971 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:12.826974 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:12.826985 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:12.826989 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:12.827030 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:12.827080 (+ 50us) server_negotiation.cc:299] Negotiation successful
0924 21:16:12.827098 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260924 21:16:12.828745 26665 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35192:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:16:12.830330 26665 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:16:12.836432 27123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:12.832416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:34241 (local address 127.0.0.1:40538)
0924 21:16:12.832596 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:12.832634 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:12.832718 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:12.832955 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:12.832958 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:12.832960 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:12.833023 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:12.833025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.834086 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.834086 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:12.835197 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:12.835200 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.836157 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.836157 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:12.836211 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:12.836220 (+ 9us) client_negotiation.cc:769] Sending connection context
0924 21:16:12.836302 (+ 82us) client_negotiation.cc:240] Negotiation successful
0924 21:16:12.836359 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":76,"threads_started":1}
I20260924 21:16:12.836444 27125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:12.832525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40538 (local address 127.6.210.129:34241)
0924 21:16:12.832728 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:12.832742 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:12.832761 (+ 19us) server_negotiation.cc:407] Connection header received
0924 21:16:12.832827 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:12.832829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:12.832844 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:12.832942 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:12.833082 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.833995 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.835309 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.836008 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.836044 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:12.836108 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:12.836320 (+ 212us) server_negotiation.cc:299] Negotiation successful
0924 21:16:12.836359 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":98,"threads_started":1}
I20260924 21:16:12.837023 27124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:12.832416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.130:38993 (local address 127.0.0.1:39550)
0924 21:16:12.832826 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:12.832842 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:12.832875 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:12.833445 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:12.833446 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:12.833449 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:12.833490 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:12.833491 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.834797 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.834798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:12.835750 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:12.835752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.836778 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.836779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:12.836837 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:12.836841 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:16:12.836902 (+ 61us) client_negotiation.cc:240] Negotiation successful
0924 21:16:12.836949 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"mutex_wait_us":3,"thread_start_us":72,"threads_started":1}
I20260924 21:16:12.837050 27126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:12.832607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39550 (local address 127.6.210.130:38993)
0924 21:16:12.833237 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:12.833242 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:12.833252 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:16:12.833290 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:12.833293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:12.833307 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:12.833380 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:12.833572 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.834721 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.835885 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.836576 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.836647 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:12.836744 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:12.836932 (+ 188us) server_negotiation.cc:299] Negotiation successful
0924 21:16:12.836974 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":76,"threads_started":1}
I20260924 21:16:12.837217 26787 tablet_service.cc:1467] Processing CreateTablet for tablet 3e44fb7f61b74abf9c92ea217cd5420c (DEFAULT_TABLE table=test-table [id=5e389b0302684b90b829e3ba169ab4b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:12.837641 26787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e44fb7f61b74abf9c92ea217cd5420c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:12.837764 27127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:12.832621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:34851 (local address 127.0.0.1:38856)
0924 21:16:12.833045 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:12.833055 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:12.833115 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:12.833347 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:12.833347 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:12.833349 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:12.833391 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:12.833392 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.834687 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.834687 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:12.835737 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:12.835740 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.837514 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.837515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:12.837577 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:12.837580 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:16:12.837659 (+ 79us) client_negotiation.cc:240] Negotiation successful
0924 21:16:12.837700 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":69,"threads_started":1}
I20260924 21:16:12.837803 27128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:12.832833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38856 (local address 127.6.210.131:34851)
0924 21:16:12.833014 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:12.833019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:12.833123 (+ 104us) server_negotiation.cc:407] Connection header received
0924 21:16:12.833184 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:12.833187 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:12.833204 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:12.833278 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:12.833448 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.834600 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.835829 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.836914 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.836970 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:12.837052 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:12.837683 (+ 631us) server_negotiation.cc:299] Negotiation successful
0924 21:16:12.837723 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":76,"threads_started":1}
I20260924 21:16:12.838503 27049 tablet_service.cc:1467] Processing CreateTablet for tablet 3e44fb7f61b74abf9c92ea217cd5420c (DEFAULT_TABLE table=test-table [id=5e389b0302684b90b829e3ba169ab4b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:12.838831 27049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e44fb7f61b74abf9c92ea217cd5420c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:12.838716 26918 tablet_service.cc:1467] Processing CreateTablet for tablet 3e44fb7f61b74abf9c92ea217cd5420c (DEFAULT_TABLE table=test-table [id=5e389b0302684b90b829e3ba169ab4b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:12.839054 26918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e44fb7f61b74abf9c92ea217cd5420c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:12.841156 27129 tablet_bootstrap.cc:492] T 3e44fb7f61b74abf9c92ea217cd5420c P 32964048b2fb4b4792a6954e5d71b997: Bootstrap starting.
I20260924 21:16:12.841419 27130 tablet_bootstrap.cc:492] T 3e44fb7f61b74abf9c92ea217cd5420c P 3b1ee242da02445485cfc9be19fc625e: Bootstrap starting.
I20260924 21:16:12.841899 27129 tablet_bootstrap.cc:654] T 3e44fb7f61b74abf9c92ea217cd5420c P 32964048b2fb4b4792a6954e5d71b997: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:12.842087 27130 tablet_bootstrap.cc:654] T 3e44fb7f61b74abf9c92ea217cd5420c P 3b1ee242da02445485cfc9be19fc625e: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:12.842309 27129 log.cc:824] T 3e44fb7f61b74abf9c92ea217cd5420c P 32964048b2fb4b4792a6954e5d71b997: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:12.842506 27130 log.cc:824] T 3e44fb7f61b74abf9c92ea217cd5420c P 3b1ee242da02445485cfc9be19fc625e: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:12.842630 27131 tablet_bootstrap.cc:492] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4: Bootstrap starting.
I20260924 21:16:12.843070 27129 tablet_bootstrap.cc:492] T 3e44fb7f61b74abf9c92ea217cd5420c P 32964048b2fb4b4792a6954e5d71b997: No bootstrap required, opened a new log
I20260924 21:16:12.843135 27130 tablet_bootstrap.cc:492] T 3e44fb7f61b74abf9c92ea217cd5420c P 3b1ee242da02445485cfc9be19fc625e: No bootstrap required, opened a new log
I20260924 21:16:12.843142 27129 ts_tablet_manager.cc:1397] T 3e44fb7f61b74abf9c92ea217cd5420c P 32964048b2fb4b4792a6954e5d71b997: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:16:12.843191 27130 ts_tablet_manager.cc:1397] T 3e44fb7f61b74abf9c92ea217cd5420c P 3b1ee242da02445485cfc9be19fc625e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 21:16:12.843202 27131 tablet_bootstrap.cc:654] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:12.843457 27131 log.cc:824] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:12.844111 27131 tablet_bootstrap.cc:492] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4: No bootstrap required, opened a new log
I20260924 21:16:12.844190 27131 ts_tablet_manager.cc:1397] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:16:12.845728 27129 raft_consensus.cc:348] T 3e44fb7f61b74abf9c92ea217cd5420c P 32964048b2fb4b4792a6954e5d71b997 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e0cf764edb439aa9a65e7683ea3ae4" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38993 } } peers { permanent_uuid: "32964048b2fb4b4792a6954e5d71b997" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 34241 } } peers { permanent_uuid: "3b1ee242da02445485cfc9be19fc625e" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 34851 } }
I20260924 21:16:12.845728 27130 raft_consensus.cc:348] T 3e44fb7f61b74abf9c92ea217cd5420c P 3b1ee242da02445485cfc9be19fc625e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e0cf764edb439aa9a65e7683ea3ae4" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38993 } } peers { permanent_uuid: "32964048b2fb4b4792a6954e5d71b997" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 34241 } } peers { permanent_uuid: "3b1ee242da02445485cfc9be19fc625e" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 34851 } }
I20260924 21:16:12.845906 27129 raft_consensus.cc:374] T 3e44fb7f61b74abf9c92ea217cd5420c P 32964048b2fb4b4792a6954e5d71b997 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:12.845906 27130 raft_consensus.cc:374] T 3e44fb7f61b74abf9c92ea217cd5420c P 3b1ee242da02445485cfc9be19fc625e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:12.845942 27130 raft_consensus.cc:729] T 3e44fb7f61b74abf9c92ea217cd5420c P 3b1ee242da02445485cfc9be19fc625e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b1ee242da02445485cfc9be19fc625e, State: Initialized, Role: FOLLOWER
I20260924 21:16:12.845942 27129 raft_consensus.cc:729] T 3e44fb7f61b74abf9c92ea217cd5420c P 32964048b2fb4b4792a6954e5d71b997 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32964048b2fb4b4792a6954e5d71b997, State: Initialized, Role: FOLLOWER
I20260924 21:16:12.846091 27130 consensus_queue.cc:260] T 3e44fb7f61b74abf9c92ea217cd5420c P 3b1ee242da02445485cfc9be19fc625e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e0cf764edb439aa9a65e7683ea3ae4" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38993 } } peers { permanent_uuid: "32964048b2fb4b4792a6954e5d71b997" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 34241 } } peers { permanent_uuid: "3b1ee242da02445485cfc9be19fc625e" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 34851 } }
I20260924 21:16:12.846086 27129 consensus_queue.cc:260] T 3e44fb7f61b74abf9c92ea217cd5420c P 32964048b2fb4b4792a6954e5d71b997 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e0cf764edb439aa9a65e7683ea3ae4" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38993 } } peers { permanent_uuid: "32964048b2fb4b4792a6954e5d71b997" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 34241 } } peers { permanent_uuid: "3b1ee242da02445485cfc9be19fc625e" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 34851 } }
I20260924 21:16:12.846374 27130 ts_tablet_manager.cc:1428] T 3e44fb7f61b74abf9c92ea217cd5420c P 3b1ee242da02445485cfc9be19fc625e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260924 21:16:12.846374 27129 ts_tablet_manager.cc:1428] T 3e44fb7f61b74abf9c92ea217cd5420c P 32964048b2fb4b4792a6954e5d71b997: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:16:12.846494 26853 heartbeater.cc:503] Master 127.6.210.190:42911 was elected leader, sending a full tablet report...
I20260924 21:16:12.846494 27115 heartbeater.cc:503] Master 127.6.210.190:42911 was elected leader, sending a full tablet report...
I20260924 21:16:12.846596 27131 raft_consensus.cc:348] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e0cf764edb439aa9a65e7683ea3ae4" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38993 } } peers { permanent_uuid: "32964048b2fb4b4792a6954e5d71b997" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 34241 } } peers { permanent_uuid: "3b1ee242da02445485cfc9be19fc625e" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 34851 } }
I20260924 21:16:12.846767 27131 raft_consensus.cc:374] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:12.846817 27131 raft_consensus.cc:729] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82e0cf764edb439aa9a65e7683ea3ae4, State: Initialized, Role: FOLLOWER
I20260924 21:16:12.846941 27131 consensus_queue.cc:260] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e0cf764edb439aa9a65e7683ea3ae4" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38993 } } peers { permanent_uuid: "32964048b2fb4b4792a6954e5d71b997" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 34241 } } peers { permanent_uuid: "3b1ee242da02445485cfc9be19fc625e" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 34851 } }
I20260924 21:16:12.847201 27131 ts_tablet_manager.cc:1428] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:16:12.847318 26984 heartbeater.cc:503] Master 127.6.210.190:42911 was elected leader, sending a full tablet report...
I20260924 21:16:12.924437 27137 raft_consensus.cc:482] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:16:12.924608 27137 raft_consensus.cc:504] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e0cf764edb439aa9a65e7683ea3ae4" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38993 } } peers { permanent_uuid: "32964048b2fb4b4792a6954e5d71b997" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 34241 } } peers { permanent_uuid: "3b1ee242da02445485cfc9be19fc625e" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 34851 } }
I20260924 21:16:12.924990 27137 leader_election.cc:288] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32964048b2fb4b4792a6954e5d71b997 (127.6.210.129:34241), 3b1ee242da02445485cfc9be19fc625e (127.6.210.131:34851)
W20260924 21:16:12.925863 26985 tablet.cc:2369] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:16:12.928109 27128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:12.925400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:56753 (local address 127.6.210.131:34851)
0924 21:16:12.925446 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:12.925448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:12.925558 (+ 110us) server_negotiation.cc:407] Connection header received
0924 21:16:12.925598 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:12.925599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:12.925608 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:12.925678 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:12.925838 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.926394 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.927396 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.927751 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.927830 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:12.927838 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:12.928002 (+ 164us) server_negotiation.cc:299] Negotiation successful
0924 21:16:12.928037 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 21:16:12.928120 27125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:12.925320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.210.130:53875 (local address 127.6.210.129:34241)
0924 21:16:12.925418 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:12.925420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:12.925429 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:16:12.925458 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:12.925459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:12.925470 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:12.925542 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:12.925712 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.926433 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.927385 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.927756 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.927849 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:12.927857 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:12.928005 (+ 148us) server_negotiation.cc:299] Negotiation successful
0924 21:16:12.928043 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260924 21:16:12.928124 26986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:12.925183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:34241 (local address 127.6.210.130:53875)
0924 21:16:12.925242 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:12.925255 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:12.925302 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:12.925545 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:12.925546 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:12.925547 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:12.925636 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:12.925638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.926515 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.926516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:12.927290 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:12.927292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.927849 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.927850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:12.927923 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:12.927926 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:16:12.927989 (+ 63us) client_negotiation.cc:240] Negotiation successful
0924 21:16:12.928036 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":9}
I20260924 21:16:12.928135 27138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:12.925287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.131:34851 (local address 127.6.210.130:56753)
0924 21:16:12.925474 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:12.925483 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:12.925544 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:12.925683 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:12.925684 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:12.925685 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:12.925755 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:12.925756 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.926474 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.926475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:12.927305 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:12.927307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.927841 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.927842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:12.927928 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:12.927932 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:16:12.927997 (+ 65us) client_negotiation.cc:240] Negotiation successful
0924 21:16:12.928034 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":43,"thread_start_us":101,"threads_started":1}
I20260924 21:16:12.928536 26807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e44fb7f61b74abf9c92ea217cd5420c" candidate_uuid: "82e0cf764edb439aa9a65e7683ea3ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32964048b2fb4b4792a6954e5d71b997" is_pre_election: true
I20260924 21:16:12.928714 26807 raft_consensus.cc:2454] T 3e44fb7f61b74abf9c92ea217cd5420c P 32964048b2fb4b4792a6954e5d71b997 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82e0cf764edb439aa9a65e7683ea3ae4 in term 0.
I20260924 21:16:12.928762 27069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e44fb7f61b74abf9c92ea217cd5420c" candidate_uuid: "82e0cf764edb439aa9a65e7683ea3ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b1ee242da02445485cfc9be19fc625e" is_pre_election: true
I20260924 21:16:12.928905 27069 raft_consensus.cc:2454] T 3e44fb7f61b74abf9c92ea217cd5420c P 3b1ee242da02445485cfc9be19fc625e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82e0cf764edb439aa9a65e7683ea3ae4 in term 0.
I20260924 21:16:12.928977 26873 leader_election.cc:302] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32964048b2fb4b4792a6954e5d71b997, 82e0cf764edb439aa9a65e7683ea3ae4; no voters:
I20260924 21:16:12.929136 27137 raft_consensus.cc:2781] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:16:12.929214 27137 raft_consensus.cc:482] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:16:12.929240 27137 raft_consensus.cc:3037] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:12.929932 27137 raft_consensus.cc:504] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e0cf764edb439aa9a65e7683ea3ae4" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38993 } } peers { permanent_uuid: "32964048b2fb4b4792a6954e5d71b997" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 34241 } } peers { permanent_uuid: "3b1ee242da02445485cfc9be19fc625e" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 34851 } }
I20260924 21:16:12.930065 27137 leader_election.cc:288] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4 [CANDIDATE]: Term 1 election: Requested vote from peers 32964048b2fb4b4792a6954e5d71b997 (127.6.210.129:34241), 3b1ee242da02445485cfc9be19fc625e (127.6.210.131:34851)
I20260924 21:16:12.930215 26807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e44fb7f61b74abf9c92ea217cd5420c" candidate_uuid: "82e0cf764edb439aa9a65e7683ea3ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32964048b2fb4b4792a6954e5d71b997"
I20260924 21:16:12.930308 27069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e44fb7f61b74abf9c92ea217cd5420c" candidate_uuid: "82e0cf764edb439aa9a65e7683ea3ae4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b1ee242da02445485cfc9be19fc625e"
I20260924 21:16:12.930380 27069 raft_consensus.cc:3037] T 3e44fb7f61b74abf9c92ea217cd5420c P 3b1ee242da02445485cfc9be19fc625e [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:12.930361 26807 raft_consensus.cc:3037] T 3e44fb7f61b74abf9c92ea217cd5420c P 32964048b2fb4b4792a6954e5d71b997 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:12.931291 26807 raft_consensus.cc:2454] T 3e44fb7f61b74abf9c92ea217cd5420c P 32964048b2fb4b4792a6954e5d71b997 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82e0cf764edb439aa9a65e7683ea3ae4 in term 1.
I20260924 21:16:12.931281 27069 raft_consensus.cc:2454] T 3e44fb7f61b74abf9c92ea217cd5420c P 3b1ee242da02445485cfc9be19fc625e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82e0cf764edb439aa9a65e7683ea3ae4 in term 1.
I20260924 21:16:12.931465 26873 leader_election.cc:302] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32964048b2fb4b4792a6954e5d71b997, 82e0cf764edb439aa9a65e7683ea3ae4; no voters:
I20260924 21:16:12.931563 27137 raft_consensus.cc:2781] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:12.931722 27137 raft_consensus.cc:686] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4 [term 1 LEADER]: Becoming Leader. State: Replica: 82e0cf764edb439aa9a65e7683ea3ae4, State: Running, Role: LEADER
I20260924 21:16:12.931828 27137 consensus_queue.cc:237] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e0cf764edb439aa9a65e7683ea3ae4" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38993 } } peers { permanent_uuid: "32964048b2fb4b4792a6954e5d71b997" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 34241 } } peers { permanent_uuid: "3b1ee242da02445485cfc9be19fc625e" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 34851 } }
I20260924 21:16:12.932713 26664 catalog_manager.cc:5495] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 82e0cf764edb439aa9a65e7683ea3ae4 (127.6.210.130). New cstate: current_term: 1 leader_uuid: "82e0cf764edb439aa9a65e7683ea3ae4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82e0cf764edb439aa9a65e7683ea3ae4" member_type: VOTER last_known_addr { host: "127.6.210.130" port: 38993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32964048b2fb4b4792a6954e5d71b997" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 34241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b1ee242da02445485cfc9be19fc625e" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 34851 } health_report { overall_health: UNKNOWN } } }
I20260924 21:16:12.952978 27126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:12.950681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39554 (local address 127.6.210.130:38993)
0924 21:16:12.950739 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:12.950742 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:12.950750 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:16:12.950833 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:12.950833 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:12.950850 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:12.950911 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:16:12.951033 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.951498 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.952260 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:12.952457 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:12.952509 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:12.952676 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:16:12.952794 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:16:12.952860 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:12.952891 (+ 31us) server_negotiation.cc:299] Negotiation successful
0924 21:16:12.952906 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 21:16:12.955025 26807 raft_consensus.cc:1261] T 3e44fb7f61b74abf9c92ea217cd5420c P 32964048b2fb4b4792a6954e5d71b997 [term 1 FOLLOWER]: Refusing update from remote peer 82e0cf764edb439aa9a65e7683ea3ae4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:16:12.955032 27069 raft_consensus.cc:1261] T 3e44fb7f61b74abf9c92ea217cd5420c P 3b1ee242da02445485cfc9be19fc625e [term 1 FOLLOWER]: Refusing update from remote peer 82e0cf764edb439aa9a65e7683ea3ae4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:16:12.955438 27139 consensus_queue.cc:1035] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b1ee242da02445485cfc9be19fc625e" member_type: VOTER last_known_addr { host: "127.6.210.131" port: 34851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:16:12.955576 27137 consensus_queue.cc:1035] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32964048b2fb4b4792a6954e5d71b997" member_type: VOTER last_known_addr { host: "127.6.210.129" port: 34241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:16:12.957826 27142 mvcc.cc:205] Tried to move back new op lower bound from 7333005610820382720 to 7333005610729308160. Current Snapshot: MvccSnapshot[applied={T|T < 7333005610820382720}]
I20260924 21:16:12.958112 27143 mvcc.cc:205] Tried to move back new op lower bound from 7333005610820382720 to 7333005610729308160. Current Snapshot: MvccSnapshot[applied={T|T < 7333005610820382720}]
I20260924 21:16:12.959693 27145 mvcc.cc:205] Tried to move back new op lower bound from 7333005610820382720 to 7333005610729308160. Current Snapshot: MvccSnapshot[applied={T|T < 7333005610820382720}]
I20260924 21:16:12.960758 26664 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:16:12.960855 26664 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:16:12.961673 26664 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bdd47bc119614d2e8359bbf575c9295c: Sending DeleteTablet for 3 replicas of tablet 3e44fb7f61b74abf9c92ea217cd5420c
I20260924 21:16:12.962026 26918 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e44fb7f61b74abf9c92ea217cd5420c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:12 UTC) from {username='slave'} at 127.0.0.1:39550
I20260924 21:16:12.962046 26787 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e44fb7f61b74abf9c92ea217cd5420c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:12 UTC) from {username='slave'} at 127.0.0.1:40538
I20260924 21:16:12.962071 27049 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e44fb7f61b74abf9c92ea217cd5420c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:12 UTC) from {username='slave'} at 127.0.0.1:38856
I20260924 21:16:12.962292 27154 tablet_replica.cc:330] T 3e44fb7f61b74abf9c92ea217cd5420c P 3b1ee242da02445485cfc9be19fc625e: stopping tablet replica
I20260924 21:16:12.962400 27154 raft_consensus.cc:2229] T 3e44fb7f61b74abf9c92ea217cd5420c P 3b1ee242da02445485cfc9be19fc625e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:16:12.962481 27152 tablet_replica.cc:330] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4: stopping tablet replica
I20260924 21:16:12.962508 27154 raft_consensus.cc:2258] T 3e44fb7f61b74abf9c92ea217cd5420c P 3b1ee242da02445485cfc9be19fc625e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:16:12.962555 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 26725
I20260924 21:16:12.962571 27152 raft_consensus.cc:2229] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4 [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:16:12.963181 27152 raft_consensus.cc:2258] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:16:12.963610 27152 ts_tablet_manager.cc:1905] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:16:12.965397 27152 ts_tablet_manager.cc:1918] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:16:12.965477 27152 log.cc:1196] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-1/wal/wals/3e44fb7f61b74abf9c92ea217cd5420c
I20260924 21:16:12.965725 27152 ts_tablet_manager.cc:1939] T 3e44fb7f61b74abf9c92ea217cd5420c P 82e0cf764edb439aa9a65e7683ea3ae4: Deleting consensus metadata
I20260924 21:16:12.966089 26652 catalog_manager.cc:4841] TS 82e0cf764edb439aa9a65e7683ea3ae4 (127.6.210.130:38993): tablet 3e44fb7f61b74abf9c92ea217cd5420c (table test-table [id=5e389b0302684b90b829e3ba169ab4b0]) successfully deleted
I20260924 21:16:12.966470 27154 ts_tablet_manager.cc:1905] T 3e44fb7f61b74abf9c92ea217cd5420c P 3b1ee242da02445485cfc9be19fc625e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:16:12.967769 27154 ts_tablet_manager.cc:1918] T 3e44fb7f61b74abf9c92ea217cd5420c P 3b1ee242da02445485cfc9be19fc625e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:16:12.967849 27154 log.cc:1196] T 3e44fb7f61b74abf9c92ea217cd5420c P 3b1ee242da02445485cfc9be19fc625e: Deleting WAL directory at /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284333163253-6986-0/minicluster-data/ts-2/wal/wals/3e44fb7f61b74abf9c92ea217cd5420c
I20260924 21:16:12.968122 27154 ts_tablet_manager.cc:1939] T 3e44fb7f61b74abf9c92ea217cd5420c P 3b1ee242da02445485cfc9be19fc625e: Deleting consensus metadata
I20260924 21:16:12.968502 26651 catalog_manager.cc:4841] TS 3b1ee242da02445485cfc9be19fc625e (127.6.210.131:34851): tablet 3e44fb7f61b74abf9c92ea217cd5420c (table test-table [id=5e389b0302684b90b829e3ba169ab4b0]) successfully deleted
W20260924 21:16:12.969393 26651 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.210.129:34241 (error 108)
I20260924 21:16:12.969642 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 26856
I20260924 21:16:12.970347 27127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:12.969703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.129:34241 (local address 127.0.0.1:40548)
0924 21:16:12.970242 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:12.970307 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.129:34241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":501}
W20260924 21:16:12.972432 26651 catalog_manager.cc:4584] TS 32964048b2fb4b4792a6954e5d71b997 (127.6.210.129:34241): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3e44fb7f61b74abf9c92ea217cd5420c: Network error: Client connection negotiation failed: client connection to 127.6.210.129:34241: connect: Connection refused (error 111)
I20260924 21:16:12.975790 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 26987
I20260924 21:16:12.980257 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 26634
2026-09-24T21:16:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (744 ms)
[----------] 6 tests from EncryptionPolicyTest (8532 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:16:13 dist-test-slave-q276 krb5kdc[27158](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:16:13 dist-test-slave-q276 krb5kdc[27158](info): set up 2 sockets
Sep 24 21:16:13 dist-test-slave-q276 krb5kdc[27158](info): commencing operation
krb5kdc: starting...
W20260924 21:16:14.118338 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.004s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:16:14 dist-test-slave-q276 krb5kdc[27158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:16:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:16:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 21:16:14.237399 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44121
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:14.318171 27171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:14.318403 27171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:14.318426 27171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:14.319654 27171 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:14.319703 27171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:14.319717 27171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:14.319731 27171 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:14.319742 27171 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:14.321257 27171 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44121
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:14.321509 27171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:14.321776 27171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:14.326730 27177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:14.326745 27176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:14.326876 27171 server_base.cc:1034] running on GCE node
W20260924 21:16:14.326722 27179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:14.327353 27171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:14.327698 27171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:14.328863 27171 hybrid_clock.cc:648] HybridClock initialized: now 1790284574328861 us; error 38 us; skew 500 ppm
Sep 24 21:16:14 dist-test-slave-q276 krb5kdc[27158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284574, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:14.337457 27171 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 21:16:14.337869 27171 webserver.cc:460] Webserver started at http://127.0.0.1:38779/ using document root <none> and password file <none>
I20260924 21:16:14.338053 27171 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:14.338093 27171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:14.338181 27171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:14.339304 27171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "ce164699e2904cfd86696138ef847d0c"
format_stamp: "Formatted at 2026-09-24 21:16:14 on dist-test-slave-q276"
server_key: "8c284ece5660053c78899786e01d6c66"
server_key_iv: "a220dab650bbe34b3d2d32b476976f65"
server_key_version: "encryptionkey@0"
I20260924 21:16:14.339607 27171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "ce164699e2904cfd86696138ef847d0c"
format_stamp: "Formatted at 2026-09-24 21:16:14 on dist-test-slave-q276"
server_key: "8c284ece5660053c78899786e01d6c66"
server_key_iv: "a220dab650bbe34b3d2d32b476976f65"
server_key_version: "encryptionkey@0"
I20260924 21:16:14.340922 27171 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 21:16:14.341765 27186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:14.341925 27171 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:16:14.342016 27171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "ce164699e2904cfd86696138ef847d0c"
format_stamp: "Formatted at 2026-09-24 21:16:14 on dist-test-slave-q276"
server_key: "8c284ece5660053c78899786e01d6c66"
server_key_iv: "a220dab650bbe34b3d2d32b476976f65"
server_key_version: "encryptionkey@0"
I20260924 21:16:14.342073 27171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:14.366793 27171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:14.368724 27171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:14.368834 27171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:14.372802 27171 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44121
I20260924 21:16:14.372867 27238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44121 every 8 connection(s)
I20260924 21:16:14.373296 27171 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:16:14.374467 27239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:14.376691 27239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce164699e2904cfd86696138ef847d0c: Bootstrap starting.
I20260924 21:16:14.377293 27239 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce164699e2904cfd86696138ef847d0c: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:14.377595 27239 log.cc:824] T 00000000000000000000000000000000 P ce164699e2904cfd86696138ef847d0c: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:14.378328 27239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce164699e2904cfd86696138ef847d0c: No bootstrap required, opened a new log
I20260924 21:16:14.379912 27239 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce164699e2904cfd86696138ef847d0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce164699e2904cfd86696138ef847d0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44121 } }
I20260924 21:16:14.380048 27239 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce164699e2904cfd86696138ef847d0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:14.380070 27239 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce164699e2904cfd86696138ef847d0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce164699e2904cfd86696138ef847d0c, State: Initialized, Role: FOLLOWER
I20260924 21:16:14.380234 27239 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce164699e2904cfd86696138ef847d0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce164699e2904cfd86696138ef847d0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44121 } }
I20260924 21:16:14.380317 27239 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce164699e2904cfd86696138ef847d0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:16:14.380362 27239 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce164699e2904cfd86696138ef847d0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:16:14.380410 27239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce164699e2904cfd86696138ef847d0c [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:14.380916 27239 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce164699e2904cfd86696138ef847d0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce164699e2904cfd86696138ef847d0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44121 } }
I20260924 21:16:14.380999 27239 leader_election.cc:302] T 00000000000000000000000000000000 P ce164699e2904cfd86696138ef847d0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce164699e2904cfd86696138ef847d0c; no voters:
I20260924 21:16:14.381146 27239 leader_election.cc:288] T 00000000000000000000000000000000 P ce164699e2904cfd86696138ef847d0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:16:14.381235 27242 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce164699e2904cfd86696138ef847d0c [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:14.381422 27239 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce164699e2904cfd86696138ef847d0c [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:14.381431 27242 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce164699e2904cfd86696138ef847d0c [term 1 LEADER]: Becoming Leader. State: Replica: ce164699e2904cfd86696138ef847d0c, State: Running, Role: LEADER
I20260924 21:16:14.381578 27242 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce164699e2904cfd86696138ef847d0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce164699e2904cfd86696138ef847d0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44121 } }
I20260924 21:16:14.381989 27244 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce164699e2904cfd86696138ef847d0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce164699e2904cfd86696138ef847d0c. Latest consensus state: current_term: 1 leader_uuid: "ce164699e2904cfd86696138ef847d0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce164699e2904cfd86696138ef847d0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44121 } } }
I20260924 21:16:14.382076 27244 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce164699e2904cfd86696138ef847d0c [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:14.382153 27243 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce164699e2904cfd86696138ef847d0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce164699e2904cfd86696138ef847d0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce164699e2904cfd86696138ef847d0c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44121 } } }
I20260924 21:16:14.382242 27243 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce164699e2904cfd86696138ef847d0c [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:14.382326 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 27171
I20260924 21:16:14.382459 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:16:14.382644 6986 external_mini_cluster.cc:1428] Setting key a60264e47c4a2f1652a3bdacca37464c
I20260924 21:16:14.382871 27251 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:16:14.383451 27251 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:16:14.386725 27251 catalog_manager.cc:1347] Generated new cluster ID: 3ee515149ce44a82b3a3c6dfeaa56699
I20260924 21:16:14.386785 27251 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 21:16:14 dist-test-slave-q276 krb5kdc[27158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260924 21:16:14.393399 27257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:14.383169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56937 (local address 127.0.0.1:44121)
0924 21:16:14.383339 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:14.383350 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:14.383390 (+ 40us) server_negotiation.cc:407] Connection header received
0924 21:16:14.383577 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:14.383713 (+ 136us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:14.383795 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:14.383960 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:14.384764 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:14.386298 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:14.387239 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:14.387682 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:14.387834 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:14.390104 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:14.390108 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:14.390108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:14.390121 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:14.391847 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:14.392249 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:14.392252 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:14.392252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:14.392371 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:14.392616 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:14.392617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:14.392618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:14.392857 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:14.393031 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:14.393118 (+ 87us) server_negotiation.cc:299] Negotiation successful
0924 21:16:14.393220 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":63,"threads_started":1}
I20260924 21:16:14.394492 6986 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:16:14.404996 27257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:14.398622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39147 (local address 127.0.0.1:44121)
0924 21:16:14.398662 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:14.398665 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:14.398875 (+ 210us) server_negotiation.cc:407] Connection header received
0924 21:16:14.398948 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:14.398949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:14.398960 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:14.399016 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:14.399538 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:14.400078 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:14.400792 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:14.401023 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:14.401108 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:14.402400 (+ 1292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:14.402402 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:14.402402 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:14.402409 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:14.403805 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:14.404165 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:14.404166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:14.404166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:14.404223 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:14.404460 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:14.404461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:14.404461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:14.404572 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:14.404684 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:14.404816 (+ 132us) server_negotiation.cc:299] Negotiation successful
0924 21:16:14.404879 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 21:16:14.405376 27251 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:16:14.406145 27251 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:16:14.414254 27251 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce164699e2904cfd86696138ef847d0c: Generated new TSK 0
I20260924 21:16:14.414486 27251 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:16:14.420869 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 27171
W20260924 21:16:14.425454 27156 connection.cc:535] client connection to 127.0.0.1:44121 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44121): Connection reset by peer (error 104)
2026-09-24T21:16:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1439 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:16:14 dist-test-slave-q276 krb5kdc[27270](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:16:14 dist-test-slave-q276 krb5kdc[27270](info): set up 2 sockets
Sep 24 21:16:14 dist-test-slave-q276 krb5kdc[27270](info): commencing operation
krb5kdc: starting...
W20260924 21:16:15.498760 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:16:15 dist-test-slave-q276 krb5kdc[27270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:16:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:16:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 21:16:15.617801 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40917
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40917
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:15.704201 27283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:15.704417 27283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:15.704448 27283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:15.705730 27283 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:15.705773 27283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:15.705788 27283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:15.705801 27283 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:15.705813 27283 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:15.707324 27283 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40917
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:15.707576 27283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:15.707823 27283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:15.712947 27289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:15.712956 27288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:15.712956 27291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:15.713586 27283 server_base.cc:1034] running on GCE node
I20260924 21:16:15.713846 27283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:15.714120 27283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:15.715318 27283 hybrid_clock.cc:648] HybridClock initialized: now 1790284575715307 us; error 46 us; skew 500 ppm
Sep 24 21:16:15 dist-test-slave-q276 krb5kdc[27270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284575, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:15.717804 27283 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 21:16:15.718232 27283 webserver.cc:460] Webserver started at http://127.0.0.1:33401/ using document root <none> and password file <none>
I20260924 21:16:15.718483 27283 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:15.718549 27283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:15.718698 27283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:15.719806 27283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "6c7e947492204d01b6c8ae3121eb3cf8"
format_stamp: "Formatted at 2026-09-24 21:16:15 on dist-test-slave-q276"
server_key: "00a92be82ba3f61a6572cad50401df6c"
server_key_iv: "f54be2dae4b99d5f0e52431fc6a7efb6"
server_key_version: "encryptionkey@0"
I20260924 21:16:15.720201 27283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "6c7e947492204d01b6c8ae3121eb3cf8"
format_stamp: "Formatted at 2026-09-24 21:16:15 on dist-test-slave-q276"
server_key: "00a92be82ba3f61a6572cad50401df6c"
server_key_iv: "f54be2dae4b99d5f0e52431fc6a7efb6"
server_key_version: "encryptionkey@0"
I20260924 21:16:15.721570 27283 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:16:15.722366 27298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:15.722566 27283 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:15.722663 27283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "6c7e947492204d01b6c8ae3121eb3cf8"
format_stamp: "Formatted at 2026-09-24 21:16:15 on dist-test-slave-q276"
server_key: "00a92be82ba3f61a6572cad50401df6c"
server_key_iv: "f54be2dae4b99d5f0e52431fc6a7efb6"
server_key_version: "encryptionkey@0"
I20260924 21:16:15.722752 27283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:15.742472 27283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:15.744472 27283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:15.744639 27283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:15.748739 27283 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40917
I20260924 21:16:15.748795 27350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40917 every 8 connection(s)
I20260924 21:16:15.749159 27283 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:16:15.750370 27351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:15.752960 27351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c7e947492204d01b6c8ae3121eb3cf8: Bootstrap starting.
I20260924 21:16:15.752985 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 27283
I20260924 21:16:15.753094 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:16:15.753309 6986 external_mini_cluster.cc:1428] Setting key 2a8301c20189dc304f58e0ff2e2bf546
I20260924 21:16:15.753770 27351 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c7e947492204d01b6c8ae3121eb3cf8: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:15.754137 27351 log.cc:824] T 00000000000000000000000000000000 P 6c7e947492204d01b6c8ae3121eb3cf8: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:15.754957 27351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c7e947492204d01b6c8ae3121eb3cf8: No bootstrap required, opened a new log
I20260924 21:16:15.757467 27351 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c7e947492204d01b6c8ae3121eb3cf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c7e947492204d01b6c8ae3121eb3cf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40917 } }
I20260924 21:16:15.757637 27351 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c7e947492204d01b6c8ae3121eb3cf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:15.757690 27351 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c7e947492204d01b6c8ae3121eb3cf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c7e947492204d01b6c8ae3121eb3cf8, State: Initialized, Role: FOLLOWER
I20260924 21:16:15.757829 27351 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c7e947492204d01b6c8ae3121eb3cf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c7e947492204d01b6c8ae3121eb3cf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40917 } }
I20260924 21:16:15.757910 27351 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c7e947492204d01b6c8ae3121eb3cf8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:16:15.757942 27351 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c7e947492204d01b6c8ae3121eb3cf8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:16:15.757992 27351 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c7e947492204d01b6c8ae3121eb3cf8 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:15.758776 27351 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c7e947492204d01b6c8ae3121eb3cf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c7e947492204d01b6c8ae3121eb3cf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40917 } }
I20260924 21:16:15.758919 27351 leader_election.cc:302] T 00000000000000000000000000000000 P 6c7e947492204d01b6c8ae3121eb3cf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c7e947492204d01b6c8ae3121eb3cf8; no voters:
I20260924 21:16:15.759111 27351 leader_election.cc:288] T 00000000000000000000000000000000 P 6c7e947492204d01b6c8ae3121eb3cf8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:16:15.759140 27356 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c7e947492204d01b6c8ae3121eb3cf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:15.759300 27356 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c7e947492204d01b6c8ae3121eb3cf8 [term 1 LEADER]: Becoming Leader. State: Replica: 6c7e947492204d01b6c8ae3121eb3cf8, State: Running, Role: LEADER
I20260924 21:16:15.759392 27356 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c7e947492204d01b6c8ae3121eb3cf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c7e947492204d01b6c8ae3121eb3cf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40917 } }
I20260924 21:16:15.759441 27351 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c7e947492204d01b6c8ae3121eb3cf8 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:15.759994 27354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:15.753905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53241 (local address 127.0.0.1:40917)
0924 21:16:15.754119 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:15.754129 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:15.754165 (+ 36us) server_negotiation.cc:407] Connection header received
0924 21:16:15.754867 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:15.754876 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:15.754941 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:15.755154 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:15.755325 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:15.756899 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:15.757673 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:15.758394 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:15.758518 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:15.759236 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:15.759247 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:15.759248 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:15.759457 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:15.759495 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:15.759508 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:15.759590 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:15.759756 (+ 166us) server_negotiation.cc:299] Negotiation successful
0924 21:16:15.759787 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":87,"threads_started":1}
I20260924 21:16:15.760689 27357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c7e947492204d01b6c8ae3121eb3cf8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c7e947492204d01b6c8ae3121eb3cf8. Latest consensus state: current_term: 1 leader_uuid: "6c7e947492204d01b6c8ae3121eb3cf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c7e947492204d01b6c8ae3121eb3cf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40917 } } }
I20260924 21:16:15.760771 27357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c7e947492204d01b6c8ae3121eb3cf8 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:15.761185 6986 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:16:15.761202 27358 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c7e947492204d01b6c8ae3121eb3cf8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c7e947492204d01b6c8ae3121eb3cf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c7e947492204d01b6c8ae3121eb3cf8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40917 } } }
I20260924 21:16:15.761327 27358 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c7e947492204d01b6c8ae3121eb3cf8 [sys.catalog]: This master's current role is: LEADER
W20260924 21:16:15.761380 27371 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6c7e947492204d01b6c8ae3121eb3cf8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 21:16:15.761420 27371 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 21:16:15.761493 27372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:16:15.761649 27372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:16:15.763136 27372 catalog_manager.cc:1347] Generated new cluster ID: 02784a5afaa646f39fd199c00fd20cef
I20260924 21:16:15.763203 27372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:16:15.770517 27354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:15.766618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41679 (local address 127.0.0.1:40917)
0924 21:16:15.766655 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:15.766657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:15.766807 (+ 150us) server_negotiation.cc:407] Connection header received
0924 21:16:15.766895 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:15.766896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:15.766906 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:15.766975 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:15.767167 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:15.767981 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:15.769031 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:15.769382 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:15.769448 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:15.770219 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:15.770221 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:15.770221 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:15.770226 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:15.770242 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:15.770246 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:15.770328 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:15.770410 (+ 82us) server_negotiation.cc:299] Negotiation successful
0924 21:16:15.770421 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 21:16:15.775942 27372 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:16:15.776737 27372 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:16:15.784618 27372 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c7e947492204d01b6c8ae3121eb3cf8: Generated new TSK 0
I20260924 21:16:15.784859 27372 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:16:15.789166 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 27283
W20260924 21:16:15.793124 27268 connection.cc:535] client connection to 127.0.0.1:40917 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40917): Connection reset by peer (error 104)
2026-09-24T21:16:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1367 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:16:15 dist-test-slave-q276 krb5kdc[27382](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:16:15 dist-test-slave-q276 krb5kdc[27382](info): set up 2 sockets
Sep 24 21:16:15 dist-test-slave-q276 krb5kdc[27382](info): commencing operation
krb5kdc: starting...
W20260924 21:16:16.822296 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:16:16 dist-test-slave-q276 krb5kdc[27382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:16:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:16:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 21:16:16.940913 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42855
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42855
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:17.021451 27395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:17.021698 27395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:17.021739 27395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:17.022929 27395 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:17.023001 27395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:17.023025 27395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:17.023047 27395 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:17.023075 27395 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:17.024531 27395 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42855
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:17.024806 27395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:17.025051 27395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:17.029922 27401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:17.029958 27403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:17.030081 27395 server_base.cc:1034] running on GCE node
W20260924 21:16:17.029933 27400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:17.030433 27395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:17.030745 27395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:17.031898 27395 hybrid_clock.cc:648] HybridClock initialized: now 1790284577031877 us; error 42 us; skew 500 ppm
Sep 24 21:16:17 dist-test-slave-q276 krb5kdc[27382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284577, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:17.034497 27395 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 21:16:17.034943 27395 webserver.cc:460] Webserver started at http://127.0.0.1:39707/ using document root <none> and password file <none>
I20260924 21:16:17.035145 27395 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:17.035187 27395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:17.035287 27395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:17.036370 27395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "f97acb74b51f4d0fab34115428a3dd76"
format_stamp: "Formatted at 2026-09-24 21:16:17 on dist-test-slave-q276"
server_key: "5c414e406222027f8e78fa86be5e1ba4"
server_key_iv: "202a0ef339e81c2b15d1ac0e21310f9a"
server_key_version: "encryptionkey@0"
I20260924 21:16:17.036720 27395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "f97acb74b51f4d0fab34115428a3dd76"
format_stamp: "Formatted at 2026-09-24 21:16:17 on dist-test-slave-q276"
server_key: "5c414e406222027f8e78fa86be5e1ba4"
server_key_iv: "202a0ef339e81c2b15d1ac0e21310f9a"
server_key_version: "encryptionkey@0"
I20260924 21:16:17.038085 27395 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260924 21:16:17.038926 27410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:17.039119 27395 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:16:17.039199 27395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "f97acb74b51f4d0fab34115428a3dd76"
format_stamp: "Formatted at 2026-09-24 21:16:17 on dist-test-slave-q276"
server_key: "5c414e406222027f8e78fa86be5e1ba4"
server_key_iv: "202a0ef339e81c2b15d1ac0e21310f9a"
server_key_version: "encryptionkey@0"
I20260924 21:16:17.039252 27395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:17.042477 27395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:17.044296 27395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:17.044399 27395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:17.048421 27395 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42855
I20260924 21:16:17.048482 27462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42855 every 8 connection(s)
I20260924 21:16:17.048830 27395 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:16:17.049873 27463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:17.052155 27463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f97acb74b51f4d0fab34115428a3dd76: Bootstrap starting.
I20260924 21:16:17.052731 27463 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f97acb74b51f4d0fab34115428a3dd76: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:17.053014 27463 log.cc:824] T 00000000000000000000000000000000 P f97acb74b51f4d0fab34115428a3dd76: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:17.053742 27463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f97acb74b51f4d0fab34115428a3dd76: No bootstrap required, opened a new log
I20260924 21:16:17.055482 27463 raft_consensus.cc:348] T 00000000000000000000000000000000 P f97acb74b51f4d0fab34115428a3dd76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97acb74b51f4d0fab34115428a3dd76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42855 } }
I20260924 21:16:17.055673 27463 raft_consensus.cc:374] T 00000000000000000000000000000000 P f97acb74b51f4d0fab34115428a3dd76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:17.055716 27463 raft_consensus.cc:729] T 00000000000000000000000000000000 P f97acb74b51f4d0fab34115428a3dd76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f97acb74b51f4d0fab34115428a3dd76, State: Initialized, Role: FOLLOWER
I20260924 21:16:17.055788 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 27395
I20260924 21:16:17.055910 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:16:17.055877 27463 consensus_queue.cc:260] T 00000000000000000000000000000000 P f97acb74b51f4d0fab34115428a3dd76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97acb74b51f4d0fab34115428a3dd76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42855 } }
I20260924 21:16:17.055961 27463 raft_consensus.cc:388] T 00000000000000000000000000000000 P f97acb74b51f4d0fab34115428a3dd76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:16:17.055994 27463 raft_consensus.cc:482] T 00000000000000000000000000000000 P f97acb74b51f4d0fab34115428a3dd76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:16:17.056038 27463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f97acb74b51f4d0fab34115428a3dd76 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:17.056102 6986 external_mini_cluster.cc:1428] Setting key 766b646a48082855a452d0ac9474318e
I20260924 21:16:17.056685 27463 raft_consensus.cc:504] T 00000000000000000000000000000000 P f97acb74b51f4d0fab34115428a3dd76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97acb74b51f4d0fab34115428a3dd76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42855 } }
I20260924 21:16:17.056797 27463 leader_election.cc:302] T 00000000000000000000000000000000 P f97acb74b51f4d0fab34115428a3dd76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f97acb74b51f4d0fab34115428a3dd76; no voters:
I20260924 21:16:17.056952 27463 leader_election.cc:288] T 00000000000000000000000000000000 P f97acb74b51f4d0fab34115428a3dd76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:16:17.057021 27468 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f97acb74b51f4d0fab34115428a3dd76 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:17.057166 27468 raft_consensus.cc:686] T 00000000000000000000000000000000 P f97acb74b51f4d0fab34115428a3dd76 [term 1 LEADER]: Becoming Leader. State: Replica: f97acb74b51f4d0fab34115428a3dd76, State: Running, Role: LEADER
I20260924 21:16:17.057211 27463 sys_catalog.cc:564] T 00000000000000000000000000000000 P f97acb74b51f4d0fab34115428a3dd76 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:17.057317 27468 consensus_queue.cc:237] T 00000000000000000000000000000000 P f97acb74b51f4d0fab34115428a3dd76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97acb74b51f4d0fab34115428a3dd76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42855 } }
I20260924 21:16:17.058715 27470 sys_catalog.cc:455] T 00000000000000000000000000000000 P f97acb74b51f4d0fab34115428a3dd76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f97acb74b51f4d0fab34115428a3dd76. Latest consensus state: current_term: 1 leader_uuid: "f97acb74b51f4d0fab34115428a3dd76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97acb74b51f4d0fab34115428a3dd76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42855 } } }
I20260924 21:16:17.058715 27469 sys_catalog.cc:455] T 00000000000000000000000000000000 P f97acb74b51f4d0fab34115428a3dd76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f97acb74b51f4d0fab34115428a3dd76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97acb74b51f4d0fab34115428a3dd76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42855 } } }
I20260924 21:16:17.058842 27470 sys_catalog.cc:458] T 00000000000000000000000000000000 P f97acb74b51f4d0fab34115428a3dd76 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:17.058844 27469 sys_catalog.cc:458] T 00000000000000000000000000000000 P f97acb74b51f4d0fab34115428a3dd76 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:17.059180 27467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:17.056630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36613 (local address 127.0.0.1:42855)
0924 21:16:17.056868 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:17.056878 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:17.056919 (+ 41us) server_negotiation.cc:407] Connection header received
0924 21:16:17.057046 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:17.057055 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:17.057086 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:17.057517 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:17.058109 (+ 592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:17.058114 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:17.058114 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:17.058120 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:17.058300 (+ 180us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:17.058324 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:17.058656 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:17.058691 (+ 35us) server_negotiation.cc:299] Negotiation successful
0924 21:16:17.058736 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":100,"threads_started":1}
I20260924 21:16:17.059360 27480 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:16:17.060177 6986 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:16:17.060267 27480 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:16:17.061877 27480 catalog_manager.cc:1347] Generated new cluster ID: 61d335f6952a4e0f86237919cc6d6287
I20260924 21:16:17.061918 27480 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:16:17.063827 27467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:17.062564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58291 (local address 127.0.0.1:42855)
0924 21:16:17.062599 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:17.062601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:17.062816 (+ 215us) server_negotiation.cc:407] Connection header received
0924 21:16:17.062882 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:17.062883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:17.062887 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:17.062948 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:17.063623 (+ 675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:17.063624 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:17.063625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:17.063627 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:17.063638 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:17.063642 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:17.063687 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:17.063747 (+ 60us) server_negotiation.cc:299] Negotiation successful
0924 21:16:17.063759 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 21:16:17.076227 27480 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:16:17.076408 27480 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:16:17.081162 27480 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f97acb74b51f4d0fab34115428a3dd76: Generated new TSK 0
I20260924 21:16:17.081339 27480 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:16:17.084621 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 27395
2026-09-24T21:16:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1295 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:16:17 dist-test-slave-q276 krb5kdc[27494](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:16:17 dist-test-slave-q276 krb5kdc[27494](info): set up 2 sockets
Sep 24 21:16:17 dist-test-slave-q276 krb5kdc[27494](info): commencing operation
krb5kdc: starting...
W20260924 21:16:18.114529 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:16:18 dist-test-slave-q276 krb5kdc[27494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:16:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:16:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 21:16:18.233208 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39641
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:18.313627 27507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:18.313832 27507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:18.313855 27507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:18.315038 27507 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:18.315079 27507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:18.315093 27507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:18.315106 27507 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:18.315160 27507 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:18.316588 27507 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:18.316826 27507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:18.317063 27507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:18.322055 27515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:18.322104 27512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:18.322055 27513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:18.322242 27507 server_base.cc:1034] running on GCE node
I20260924 21:16:18.322578 27507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:18.322861 27507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:18.324036 27507 hybrid_clock.cc:648] HybridClock initialized: now 1790284578324022 us; error 26 us; skew 500 ppm
Sep 24 21:16:18 dist-test-slave-q276 krb5kdc[27494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284578, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:18.326494 27507 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 21:16:18.326984 27507 webserver.cc:460] Webserver started at http://127.0.0.1:45789/ using document root <none> and password file <none>
I20260924 21:16:18.327219 27507 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:18.327287 27507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:18.327442 27507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:18.328500 27507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "833d38b36d4b4fcd909324813ba6b318"
format_stamp: "Formatted at 2026-09-24 21:16:18 on dist-test-slave-q276"
server_key: "233307cdacdd91ed5a6371e77099b045"
server_key_iv: "ab33e865d6a819259dcf08b4bcdd5d1e"
server_key_version: "encryptionkey@0"
I20260924 21:16:18.328859 27507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "833d38b36d4b4fcd909324813ba6b318"
format_stamp: "Formatted at 2026-09-24 21:16:18 on dist-test-slave-q276"
server_key: "233307cdacdd91ed5a6371e77099b045"
server_key_iv: "ab33e865d6a819259dcf08b4bcdd5d1e"
server_key_version: "encryptionkey@0"
I20260924 21:16:18.330173 27507 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:16:18.330961 27522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:18.331136 27507 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:16:18.331226 27507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "833d38b36d4b4fcd909324813ba6b318"
format_stamp: "Formatted at 2026-09-24 21:16:18 on dist-test-slave-q276"
server_key: "233307cdacdd91ed5a6371e77099b045"
server_key_iv: "ab33e865d6a819259dcf08b4bcdd5d1e"
server_key_version: "encryptionkey@0"
I20260924 21:16:18.331307 27507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:18.359211 27507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:18.361227 27507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:18.361400 27507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:18.365604 27507 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39641
I20260924 21:16:18.365639 27574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39641 every 8 connection(s)
I20260924 21:16:18.366026 27507 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:16:18.367093 27575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:18.368327 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 27507
I20260924 21:16:18.368459 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:16:18.368636 6986 external_mini_cluster.cc:1428] Setting key 09192de786f7bbc770495bcd5ab39a6f
I20260924 21:16:18.369845 27575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 833d38b36d4b4fcd909324813ba6b318: Bootstrap starting.
I20260924 21:16:18.370499 27575 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 833d38b36d4b4fcd909324813ba6b318: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:18.370772 27575 log.cc:824] T 00000000000000000000000000000000 P 833d38b36d4b4fcd909324813ba6b318: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:18.371431 27575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 833d38b36d4b4fcd909324813ba6b318: No bootstrap required, opened a new log
I20260924 21:16:18.373170 27575 raft_consensus.cc:348] T 00000000000000000000000000000000 P 833d38b36d4b4fcd909324813ba6b318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "833d38b36d4b4fcd909324813ba6b318" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39641 } }
I20260924 21:16:18.373363 27575 raft_consensus.cc:374] T 00000000000000000000000000000000 P 833d38b36d4b4fcd909324813ba6b318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:18.373401 27575 raft_consensus.cc:729] T 00000000000000000000000000000000 P 833d38b36d4b4fcd909324813ba6b318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 833d38b36d4b4fcd909324813ba6b318, State: Initialized, Role: FOLLOWER
I20260924 21:16:18.373545 27575 consensus_queue.cc:260] T 00000000000000000000000000000000 P 833d38b36d4b4fcd909324813ba6b318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "833d38b36d4b4fcd909324813ba6b318" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39641 } }
I20260924 21:16:18.373663 27575 raft_consensus.cc:388] T 00000000000000000000000000000000 P 833d38b36d4b4fcd909324813ba6b318 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:16:18.373714 27575 raft_consensus.cc:482] T 00000000000000000000000000000000 P 833d38b36d4b4fcd909324813ba6b318 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:16:18.373785 27575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 833d38b36d4b4fcd909324813ba6b318 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:18.374387 27575 raft_consensus.cc:504] T 00000000000000000000000000000000 P 833d38b36d4b4fcd909324813ba6b318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "833d38b36d4b4fcd909324813ba6b318" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39641 } }
I20260924 21:16:18.374500 27575 leader_election.cc:302] T 00000000000000000000000000000000 P 833d38b36d4b4fcd909324813ba6b318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 833d38b36d4b4fcd909324813ba6b318; no voters:
I20260924 21:16:18.374672 27575 leader_election.cc:288] T 00000000000000000000000000000000 P 833d38b36d4b4fcd909324813ba6b318 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:16:18.374728 27580 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 833d38b36d4b4fcd909324813ba6b318 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:18.374868 27580 raft_consensus.cc:686] T 00000000000000000000000000000000 P 833d38b36d4b4fcd909324813ba6b318 [term 1 LEADER]: Becoming Leader. State: Replica: 833d38b36d4b4fcd909324813ba6b318, State: Running, Role: LEADER
I20260924 21:16:18.374925 27575 sys_catalog.cc:564] T 00000000000000000000000000000000 P 833d38b36d4b4fcd909324813ba6b318 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:18.375000 27580 consensus_queue.cc:237] T 00000000000000000000000000000000 P 833d38b36d4b4fcd909324813ba6b318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "833d38b36d4b4fcd909324813ba6b318" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39641 } }
Sep 24 21:16:18 dist-test-slave-q276 krb5kdc[27494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260924 21:16:18.375371 27581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 833d38b36d4b4fcd909324813ba6b318 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "833d38b36d4b4fcd909324813ba6b318" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "833d38b36d4b4fcd909324813ba6b318" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39641 } } }
I20260924 21:16:18.375485 27581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 833d38b36d4b4fcd909324813ba6b318 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:18.375356 27580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 833d38b36d4b4fcd909324813ba6b318 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 833d38b36d4b4fcd909324813ba6b318. Latest consensus state: current_term: 1 leader_uuid: "833d38b36d4b4fcd909324813ba6b318" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "833d38b36d4b4fcd909324813ba6b318" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39641 } } }
I20260924 21:16:18.375694 27580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 833d38b36d4b4fcd909324813ba6b318 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:18.375705 27588 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:16:18.376575 27588 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:16:18.379431 27588 catalog_manager.cc:1347] Generated new cluster ID: e1d487ee2ca44db2a9855945fb337522
I20260924 21:16:18.379495 27588 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:16:18.379647 27578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:18.369174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56139 (local address 127.0.0.1:39641)
0924 21:16:18.369347 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:18.369356 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:18.369391 (+ 35us) server_negotiation.cc:407] Connection header received
0924 21:16:18.369799 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:18.369806 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:18.369850 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:18.370006 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:18.370714 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:18.371727 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:18.372490 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:18.372893 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:18.373028 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:18.375818 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:18.375821 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:18.375821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:18.375853 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:18.377711 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:18.378153 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:18.378156 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:18.378156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:18.378307 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:18.378548 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:18.378550 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:18.378550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:18.378873 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:18.379036 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:18.379204 (+ 168us) server_negotiation.cc:299] Negotiation successful
0924 21:16:18.379350 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":87,"threads_started":1}
I20260924 21:16:18.380570 6986 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:16:18.395360 27578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:18.381855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59617 (local address 127.0.0.1:39641)
0924 21:16:18.381891 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:18.381893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:18.388946 (+ 7053us) server_negotiation.cc:407] Connection header received
0924 21:16:18.389065 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:18.389066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:18.389085 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:18.389116 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:18.389911 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:18.390467 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:18.391166 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:18.391387 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:18.391477 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:18.392740 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:18.392741 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:18.392741 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:18.392749 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:18.394168 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:18.394582 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:18.394583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:18.394584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:18.394642 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:18.394845 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:18.394846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:18.394846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:18.394964 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:18.395065 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:18.395172 (+ 107us) server_negotiation.cc:299] Negotiation successful
0924 21:16:18.395251 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 21:16:18.415632 27588 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:16:18.416321 27588 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:16:18.422636 27588 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 833d38b36d4b4fcd909324813ba6b318: Generated new TSK 0
I20260924 21:16:18.422878 27588 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:16:18.427546 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 27507
2026-09-24T21:16:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1342 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:16:18 dist-test-slave-q276 krb5kdc[27606](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:16:18 dist-test-slave-q276 krb5kdc[27606](info): set up 2 sockets
Sep 24 21:16:18 dist-test-slave-q276 krb5kdc[27606](info): commencing operation
krb5kdc: starting...
W20260924 21:16:19.514818 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:16:19 dist-test-slave-q276 krb5kdc[27606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:16:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:16:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 21:16:19.631182 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41507
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41507
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:19.711131 27619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:19.711377 27619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:19.711417 27619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:19.712574 27619 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:19.712641 27619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:19.712666 27619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:19.712697 27619 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:19.712718 27619 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:19.714211 27619 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:19.714504 27619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:19.714758 27619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:19.719641 27627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:19.719678 27625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:19.719820 27619 server_base.cc:1034] running on GCE node
W20260924 21:16:19.719645 27624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:19.720184 27619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:19.720487 27619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:19.721635 27619 hybrid_clock.cc:648] HybridClock initialized: now 1790284579721627 us; error 33 us; skew 500 ppm
Sep 24 21:16:19 dist-test-slave-q276 krb5kdc[27606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284579, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:19.724148 27619 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 21:16:19.724617 27619 webserver.cc:460] Webserver started at http://127.0.0.1:41947/ using document root <none> and password file <none>
I20260924 21:16:19.724803 27619 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:19.724843 27619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:19.724936 27619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:19.725971 27619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "08161377a5c446f898647d85ea644431"
format_stamp: "Formatted at 2026-09-24 21:16:19 on dist-test-slave-q276"
server_key: "a2862ba610f9afb16a58f3f046145acd"
server_key_iv: "e3873cad1906dd9882abde1e8b5ff5e3"
server_key_version: "encryptionkey@0"
I20260924 21:16:19.726347 27619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "08161377a5c446f898647d85ea644431"
format_stamp: "Formatted at 2026-09-24 21:16:19 on dist-test-slave-q276"
server_key: "a2862ba610f9afb16a58f3f046145acd"
server_key_iv: "e3873cad1906dd9882abde1e8b5ff5e3"
server_key_version: "encryptionkey@0"
I20260924 21:16:19.727622 27619 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:16:19.728325 27634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:19.728525 27619 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:16:19.728613 27619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "08161377a5c446f898647d85ea644431"
format_stamp: "Formatted at 2026-09-24 21:16:19 on dist-test-slave-q276"
server_key: "a2862ba610f9afb16a58f3f046145acd"
server_key_iv: "e3873cad1906dd9882abde1e8b5ff5e3"
server_key_version: "encryptionkey@0"
I20260924 21:16:19.728667 27619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:19.751113 27619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:19.753119 27619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:19.753228 27619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:19.757215 27619 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41507
I20260924 21:16:19.757265 27686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41507 every 8 connection(s)
I20260924 21:16:19.757624 27619 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:16:19.758599 27687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:19.760823 27687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08161377a5c446f898647d85ea644431: Bootstrap starting.
I20260924 21:16:19.761399 27687 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08161377a5c446f898647d85ea644431: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:19.761685 27687 log.cc:824] T 00000000000000000000000000000000 P 08161377a5c446f898647d85ea644431: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:19.762421 27687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08161377a5c446f898647d85ea644431: No bootstrap required, opened a new log
I20260924 21:16:19.763979 27687 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08161377a5c446f898647d85ea644431 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08161377a5c446f898647d85ea644431" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41507 } }
I20260924 21:16:19.764134 27687 raft_consensus.cc:374] T 00000000000000000000000000000000 P 08161377a5c446f898647d85ea644431 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:19.764184 27687 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08161377a5c446f898647d85ea644431 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08161377a5c446f898647d85ea644431, State: Initialized, Role: FOLLOWER
I20260924 21:16:19.764351 27687 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08161377a5c446f898647d85ea644431 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08161377a5c446f898647d85ea644431" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41507 } }
I20260924 21:16:19.764432 27687 raft_consensus.cc:388] T 00000000000000000000000000000000 P 08161377a5c446f898647d85ea644431 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:16:19.764482 27687 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08161377a5c446f898647d85ea644431 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:16:19.764534 27687 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08161377a5c446f898647d85ea644431 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:19.765043 27687 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08161377a5c446f898647d85ea644431 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08161377a5c446f898647d85ea644431" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41507 } }
I20260924 21:16:19.765161 27687 leader_election.cc:302] T 00000000000000000000000000000000 P 08161377a5c446f898647d85ea644431 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 08161377a5c446f898647d85ea644431; no voters:
I20260924 21:16:19.765347 27687 leader_election.cc:288] T 00000000000000000000000000000000 P 08161377a5c446f898647d85ea644431 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:16:19.765385 27690 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08161377a5c446f898647d85ea644431 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:19.765563 27690 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08161377a5c446f898647d85ea644431 [term 1 LEADER]: Becoming Leader. State: Replica: 08161377a5c446f898647d85ea644431, State: Running, Role: LEADER
I20260924 21:16:19.765642 27687 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08161377a5c446f898647d85ea644431 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:19.765729 27690 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08161377a5c446f898647d85ea644431 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08161377a5c446f898647d85ea644431" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41507 } }
I20260924 21:16:19.766129 27692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08161377a5c446f898647d85ea644431 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08161377a5c446f898647d85ea644431. Latest consensus state: current_term: 1 leader_uuid: "08161377a5c446f898647d85ea644431" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08161377a5c446f898647d85ea644431" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41507 } } }
I20260924 21:16:19.766153 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 27619
I20260924 21:16:19.766202 27692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08161377a5c446f898647d85ea644431 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:19.766286 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:16:19.766474 6986 external_mini_cluster.cc:1428] Setting key 88ac018c3ad3859b4072d9da6c3e70e7
I20260924 21:16:19.766482 27691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08161377a5c446f898647d85ea644431 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "08161377a5c446f898647d85ea644431" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08161377a5c446f898647d85ea644431" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41507 } } }
I20260924 21:16:19.766546 27691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08161377a5c446f898647d85ea644431 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:19.766896 27694 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:16:19.767524 27694 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:16:19.769959 27694 catalog_manager.cc:1347] Generated new cluster ID: e5b07b2a4c314cf1bfc0d08bb36b4578
I20260924 21:16:19.770010 27694 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:16:19.772660 27706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:19.767119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46769 (local address 127.0.0.1:41507)
0924 21:16:19.767396 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:19.767406 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:19.767437 (+ 31us) server_negotiation.cc:407] Connection header received
0924 21:16:19.767562 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:19.767569 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:19.767624 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:19.768012 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:19.768102 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:19.769471 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:19.770513 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:19.771148 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:19.771261 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:19.771982 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:19.771987 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:19.771987 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:19.772189 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:19.772228 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:19.772243 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:19.772313 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:19.772383 (+ 70us) server_negotiation.cc:299] Negotiation successful
0924 21:16:19.772443 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":76,"threads_started":1}
I20260924 21:16:19.773624 6986 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:16:19.773813 27694 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:16:19.774508 27694 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:16:19.777815 27694 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 08161377a5c446f898647d85ea644431: Generated new TSK 0
I20260924 21:16:19.777976 27694 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:16:19.778561 27706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:19.775103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41595 (local address 127.0.0.1:41507)
0924 21:16:19.775143 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:19.775144 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:19.775226 (+ 82us) server_negotiation.cc:407] Connection header received
0924 21:16:19.775270 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:19.775271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:19.775282 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:19.775349 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:19.775482 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:19.776218 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:19.777229 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:19.777572 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:19.777642 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:19.778357 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:19.778358 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:19.778359 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:19.778363 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:19.778381 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:19.778385 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:19.778433 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:19.778495 (+ 62us) server_negotiation.cc:299] Negotiation successful
0924 21:16:19.778517 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 21:16:19.780625 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 27619
2026-09-24T21:16:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1351 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:16:19 dist-test-slave-q276 krb5kdc[27718](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:16:19 dist-test-slave-q276 krb5kdc[27718](info): set up 2 sockets
Sep 24 21:16:19 dist-test-slave-q276 krb5kdc[27718](info): commencing operation
krb5kdc: starting...
W20260924 21:16:20.810667 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:16:20 dist-test-slave-q276 krb5kdc[27718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:16:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:16:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 21:16:20.928964 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43099
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:21.010393 27731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:21.010610 27731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:21.010634 27731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:21.011763 27731 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:21.011799 27731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:21.011813 27731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:21.011826 27731 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:21.011838 27731 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:21.013315 27731 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43099
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:21.013518 27731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:21.013695 27731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:21.018561 27736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:21.018697 27731 server_base.cc:1034] running on GCE node
W20260924 21:16:21.018569 27739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:21.018657 27737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:21.019088 27731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:21.019335 27731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:21.020478 27731 hybrid_clock.cc:648] HybridClock initialized: now 1790284581020471 us; error 35 us; skew 500 ppm
Sep 24 21:16:21 dist-test-slave-q276 krb5kdc[27718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284581, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:21.022943 27731 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 21:16:21.023353 27731 webserver.cc:460] Webserver started at http://127.0.0.1:43089/ using document root <none> and password file <none>
I20260924 21:16:21.023602 27731 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:21.023672 27731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:21.023816 27731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:21.024870 27731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "78bc8e22927241a0a53d978e52821d2b"
format_stamp: "Formatted at 2026-09-24 21:16:21 on dist-test-slave-q276"
server_key: "99a6105baf517488885059ff0107bf2f"
server_key_iv: "6daab5e42c816069b0903f7b27e09575"
server_key_version: "encryptionkey@0"
I20260924 21:16:21.025238 27731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "78bc8e22927241a0a53d978e52821d2b"
format_stamp: "Formatted at 2026-09-24 21:16:21 on dist-test-slave-q276"
server_key: "99a6105baf517488885059ff0107bf2f"
server_key_iv: "6daab5e42c816069b0903f7b27e09575"
server_key_version: "encryptionkey@0"
I20260924 21:16:21.026618 27731 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:16:21.027390 27746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:21.027623 27731 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:21.027717 27731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "78bc8e22927241a0a53d978e52821d2b"
format_stamp: "Formatted at 2026-09-24 21:16:21 on dist-test-slave-q276"
server_key: "99a6105baf517488885059ff0107bf2f"
server_key_iv: "6daab5e42c816069b0903f7b27e09575"
server_key_version: "encryptionkey@0"
I20260924 21:16:21.027772 27731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:21.031169 27731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:21.033383 27731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:21.033511 27731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:21.037600 27731 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43099
I20260924 21:16:21.037659 27798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43099 every 8 connection(s)
I20260924 21:16:21.038007 27731 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:16:21.039062 27799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:21.041290 27799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78bc8e22927241a0a53d978e52821d2b: Bootstrap starting.
I20260924 21:16:21.041882 27799 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78bc8e22927241a0a53d978e52821d2b: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:21.042163 27799 log.cc:824] T 00000000000000000000000000000000 P 78bc8e22927241a0a53d978e52821d2b: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:21.042886 27799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78bc8e22927241a0a53d978e52821d2b: No bootstrap required, opened a new log
I20260924 21:16:21.043813 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 27731
I20260924 21:16:21.043932 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:16:21.044098 6986 external_mini_cluster.cc:1428] Setting key b38c3a71857b5ea2a27a73d52b2d9505
I20260924 21:16:21.045034 27799 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78bc8e22927241a0a53d978e52821d2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78bc8e22927241a0a53d978e52821d2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43099 } }
I20260924 21:16:21.045215 27799 raft_consensus.cc:374] T 00000000000000000000000000000000 P 78bc8e22927241a0a53d978e52821d2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:21.045255 27799 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78bc8e22927241a0a53d978e52821d2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78bc8e22927241a0a53d978e52821d2b, State: Initialized, Role: FOLLOWER
I20260924 21:16:21.045433 27799 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78bc8e22927241a0a53d978e52821d2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78bc8e22927241a0a53d978e52821d2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43099 } }
I20260924 21:16:21.045513 27799 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78bc8e22927241a0a53d978e52821d2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:16:21.045536 27799 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78bc8e22927241a0a53d978e52821d2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:16:21.045570 27799 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78bc8e22927241a0a53d978e52821d2b [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:21.046406 27803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:21.044592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40127 (local address 127.0.0.1:43099)
0924 21:16:21.044777 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:21.044785 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:21.044811 (+ 26us) server_negotiation.cc:407] Connection header received
0924 21:16:21.044991 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:21.044996 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:21.045020 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:21.045128 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:21.045896 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:21.045899 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:21.045899 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:21.045903 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:21.045946 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:21.045956 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:21.046010 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:21.046110 (+ 100us) server_negotiation.cc:299] Negotiation successful
0924 21:16:21.046163 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":102,"threads_started":1}
I20260924 21:16:21.046715 27799 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78bc8e22927241a0a53d978e52821d2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78bc8e22927241a0a53d978e52821d2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43099 } }
I20260924 21:16:21.046825 27799 leader_election.cc:302] T 00000000000000000000000000000000 P 78bc8e22927241a0a53d978e52821d2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78bc8e22927241a0a53d978e52821d2b; no voters:
I20260924 21:16:21.046983 27799 leader_election.cc:288] T 00000000000000000000000000000000 P 78bc8e22927241a0a53d978e52821d2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:16:21.047003 27804 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78bc8e22927241a0a53d978e52821d2b [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:21.047132 27804 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78bc8e22927241a0a53d978e52821d2b [term 1 LEADER]: Becoming Leader. State: Replica: 78bc8e22927241a0a53d978e52821d2b, State: Running, Role: LEADER
I20260924 21:16:21.047247 27799 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78bc8e22927241a0a53d978e52821d2b [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:21.047262 27804 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78bc8e22927241a0a53d978e52821d2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78bc8e22927241a0a53d978e52821d2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43099 } }
I20260924 21:16:21.047340 6986 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:16:21.047701 27806 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78bc8e22927241a0a53d978e52821d2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "78bc8e22927241a0a53d978e52821d2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78bc8e22927241a0a53d978e52821d2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43099 } } }
I20260924 21:16:21.047783 27806 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78bc8e22927241a0a53d978e52821d2b [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:21.047937 27805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78bc8e22927241a0a53d978e52821d2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78bc8e22927241a0a53d978e52821d2b. Latest consensus state: current_term: 1 leader_uuid: "78bc8e22927241a0a53d978e52821d2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78bc8e22927241a0a53d978e52821d2b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43099 } } }
I20260924 21:16:21.047999 27805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78bc8e22927241a0a53d978e52821d2b [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:21.048105 27812 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:16:21.048599 27812 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:16:21.055689 27812 catalog_manager.cc:1347] Generated new cluster ID: 1505daaa12524b4ab6ea5b6093a9a9d8
I20260924 21:16:21.055776 27812 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:16:21.056274 27803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:21.054805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47725 (local address 127.0.0.1:43099)
0924 21:16:21.054858 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:21.054860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:21.054982 (+ 122us) server_negotiation.cc:407] Connection header received
0924 21:16:21.055101 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:21.055103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:21.055108 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:21.055140 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:21.056009 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:21.056010 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:21.056011 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:21.056016 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:21.056033 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:21.056039 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:21.056108 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:21.056192 (+ 84us) server_negotiation.cc:299] Negotiation successful
0924 21:16:21.056208 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260924 21:16:21.064668 27812 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:16:21.064862 27812 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:16:21.075162 27812 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78bc8e22927241a0a53d978e52821d2b: Generated new TSK 0
I20260924 21:16:21.075397 27812 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:16:21.084592 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 27731
2026-09-24T21:16:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1304 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:16:21 dist-test-slave-q276 krb5kdc[27830](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:16:21 dist-test-slave-q276 krb5kdc[27830](info): set up 2 sockets
Sep 24 21:16:21 dist-test-slave-q276 krb5kdc[27830](info): commencing operation
krb5kdc: starting...
W20260924 21:16:22.118521 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:16:22 dist-test-slave-q276 krb5kdc[27830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:16:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:16:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 21:16:22.235428 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41777
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41777
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:22.315693 27843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:22.315900 27843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:22.315922 27843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:22.317022 27843 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:22.317060 27843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:22.317075 27843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:22.317086 27843 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:22.317098 27843 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:22.318511 27843 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41777
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:22.318696 27843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:22.318862 27843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:22.323592 27849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:22.323638 27851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:22.323640 27848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:22.323752 27843 server_base.cc:1034] running on GCE node
I20260924 21:16:22.324031 27843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:22.324265 27843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:22.325400 27843 hybrid_clock.cc:648] HybridClock initialized: now 1790284582325388 us; error 26 us; skew 500 ppm
Sep 24 21:16:22 dist-test-slave-q276 krb5kdc[27830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284582, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:22.327912 27843 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 21:16:22.328547 27843 webserver.cc:460] Webserver started at http://127.0.0.1:44811/ using document root <none> and password file <none>
I20260924 21:16:22.328805 27843 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:22.328879 27843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:22.329072 27843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:22.330148 27843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "710286a985f249d9bed33ec7367baec9"
format_stamp: "Formatted at 2026-09-24 21:16:22 on dist-test-slave-q276"
server_key: "9b41c2d1d84936c60dd03f11768c30e0"
server_key_iv: "5271ac02857b511c5b7031644117d89f"
server_key_version: "encryptionkey@0"
I20260924 21:16:22.330593 27843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "710286a985f249d9bed33ec7367baec9"
format_stamp: "Formatted at 2026-09-24 21:16:22 on dist-test-slave-q276"
server_key: "9b41c2d1d84936c60dd03f11768c30e0"
server_key_iv: "5271ac02857b511c5b7031644117d89f"
server_key_version: "encryptionkey@0"
I20260924 21:16:22.331899 27843 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:22.332631 27858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:22.332819 27843 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:16:22.332926 27843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "710286a985f249d9bed33ec7367baec9"
format_stamp: "Formatted at 2026-09-24 21:16:22 on dist-test-slave-q276"
server_key: "9b41c2d1d84936c60dd03f11768c30e0"
server_key_iv: "5271ac02857b511c5b7031644117d89f"
server_key_version: "encryptionkey@0"
I20260924 21:16:22.333012 27843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:22.346529 27843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:22.348266 27843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:22.348425 27843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:22.352372 27843 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41777
I20260924 21:16:22.352403 27910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41777 every 8 connection(s)
I20260924 21:16:22.352756 27843 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:16:22.353704 27911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:22.356117 27911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 710286a985f249d9bed33ec7367baec9: Bootstrap starting.
I20260924 21:16:22.356695 27911 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 710286a985f249d9bed33ec7367baec9: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:22.356981 27911 log.cc:824] T 00000000000000000000000000000000 P 710286a985f249d9bed33ec7367baec9: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:22.357674 27911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 710286a985f249d9bed33ec7367baec9: No bootstrap required, opened a new log
I20260924 21:16:22.359304 27911 raft_consensus.cc:348] T 00000000000000000000000000000000 P 710286a985f249d9bed33ec7367baec9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "710286a985f249d9bed33ec7367baec9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41777 } }
I20260924 21:16:22.359440 27911 raft_consensus.cc:374] T 00000000000000000000000000000000 P 710286a985f249d9bed33ec7367baec9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:22.359462 27911 raft_consensus.cc:729] T 00000000000000000000000000000000 P 710286a985f249d9bed33ec7367baec9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 710286a985f249d9bed33ec7367baec9, State: Initialized, Role: FOLLOWER
I20260924 21:16:22.359647 27911 consensus_queue.cc:260] T 00000000000000000000000000000000 P 710286a985f249d9bed33ec7367baec9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "710286a985f249d9bed33ec7367baec9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41777 } }
I20260924 21:16:22.359712 27911 raft_consensus.cc:388] T 00000000000000000000000000000000 P 710286a985f249d9bed33ec7367baec9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:16:22.359755 27911 raft_consensus.cc:482] T 00000000000000000000000000000000 P 710286a985f249d9bed33ec7367baec9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:16:22.359808 27911 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 710286a985f249d9bed33ec7367baec9 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:22.359943 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 27843
I20260924 21:16:22.360080 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:16:22.360255 6986 external_mini_cluster.cc:1428] Setting key b16be8fbf2631cec27fa153b5ca61aca
I20260924 21:16:22.360460 27911 raft_consensus.cc:504] T 00000000000000000000000000000000 P 710286a985f249d9bed33ec7367baec9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "710286a985f249d9bed33ec7367baec9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41777 } }
I20260924 21:16:22.360582 27911 leader_election.cc:302] T 00000000000000000000000000000000 P 710286a985f249d9bed33ec7367baec9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 710286a985f249d9bed33ec7367baec9; no voters:
I20260924 21:16:22.360743 27911 leader_election.cc:288] T 00000000000000000000000000000000 P 710286a985f249d9bed33ec7367baec9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:16:22.360785 27914 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 710286a985f249d9bed33ec7367baec9 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:22.360994 27914 raft_consensus.cc:686] T 00000000000000000000000000000000 P 710286a985f249d9bed33ec7367baec9 [term 1 LEADER]: Becoming Leader. State: Replica: 710286a985f249d9bed33ec7367baec9, State: Running, Role: LEADER
I20260924 21:16:22.361060 27911 sys_catalog.cc:564] T 00000000000000000000000000000000 P 710286a985f249d9bed33ec7367baec9 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:22.361127 27914 consensus_queue.cc:237] T 00000000000000000000000000000000 P 710286a985f249d9bed33ec7367baec9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "710286a985f249d9bed33ec7367baec9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41777 } }
I20260924 21:16:22.361577 27914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 710286a985f249d9bed33ec7367baec9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 710286a985f249d9bed33ec7367baec9. Latest consensus state: current_term: 1 leader_uuid: "710286a985f249d9bed33ec7367baec9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "710286a985f249d9bed33ec7367baec9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41777 } } }
I20260924 21:16:22.361702 27914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 710286a985f249d9bed33ec7367baec9 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:22.362208 27921 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:16:22.362743 27921 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:16:22.363377 27918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 710286a985f249d9bed33ec7367baec9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "710286a985f249d9bed33ec7367baec9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "710286a985f249d9bed33ec7367baec9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41777 } } }
I20260924 21:16:22.363503 27918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 710286a985f249d9bed33ec7367baec9 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:22.364316 27921 catalog_manager.cc:1347] Generated new cluster ID: f91e223c352e4ebc8d5effc349ad32f4
I20260924 21:16:22.364367 27921 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 21:16:22 dist-test-slave-q276 krb5kdc[27830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260924 21:16:22.372182 27916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:22.360711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:45337 (local address 127.0.0.1:41777)
0924 21:16:22.360898 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:22.360908 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:22.360944 (+ 36us) server_negotiation.cc:407] Connection header received
0924 21:16:22.361057 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:22.361064 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:22.361127 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:22.361322 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:22.362132 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:22.363526 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:22.364557 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:22.365160 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:22.365254 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:22.368157 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:22.368161 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:22.368162 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:22.368175 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:22.370420 (+ 2245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:22.370872 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:22.370876 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:22.370876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:22.370996 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:22.371243 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:22.371244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:22.371244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:22.371564 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:22.371735 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:22.371843 (+ 108us) server_negotiation.cc:299] Negotiation successful
0924 21:16:22.371945 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":64,"threads_started":1}
I20260924 21:16:22.373270 6986 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:16:22.375722 27921 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:16:22.376426 27921 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:16:22.381397 27921 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 710286a985f249d9bed33ec7367baec9: Generated new TSK 0
I20260924 21:16:22.381560 27921 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:16:22.381767 27916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:22.374635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:41781 (local address 127.0.0.1:41777)
0924 21:16:22.374680 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:22.374682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:22.375385 (+ 703us) server_negotiation.cc:407] Connection header received
0924 21:16:22.375482 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:22.375483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:22.375494 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:22.375557 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:22.376174 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:22.376688 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:22.377377 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:22.377597 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:22.377681 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:22.378939 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:22.378941 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:22.378941 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:22.378948 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:22.380366 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:22.380744 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:22.380745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:22.380745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:22.380800 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:22.381179 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:22.381180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:22.381180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:22.381310 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:22.381435 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:22.381603 (+ 168us) server_negotiation.cc:299] Negotiation successful
0924 21:16:22.381665 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":15}
I20260924 21:16:22.383802 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 27843
W20260924 21:16:22.387691 27828 connection.cc:535] client connection to 127.0.0.1:41777 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41777): Connection reset by peer (error 104)
2026-09-24T21:16:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1298 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:16:22 dist-test-slave-q276 krb5kdc[27942](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:16:22 dist-test-slave-q276 krb5kdc[27942](info): set up 2 sockets
Sep 24 21:16:22 dist-test-slave-q276 krb5kdc[27942](info): commencing operation
krb5kdc: starting...
W20260924 21:16:23.426493 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:16:23 dist-test-slave-q276 krb5kdc[27942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:16:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:16:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 21:16:23.547410 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41265
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41265
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:23.635619 27955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:23.635872 27955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:23.635912 27955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:23.637634 27955 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:23.637701 27955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:23.637724 27955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:23.637744 27955 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:23.637766 27955 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:23.640230 27955 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41265
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:23.640618 27955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:23.640908 27955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:23.647789 27963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:23.647789 27960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:23.648156 27961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:23.648838 27955 server_base.cc:1034] running on GCE node
I20260924 21:16:23.649475 27955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:23.649863 27955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:23.651059 27955 hybrid_clock.cc:648] HybridClock initialized: now 1790284583651038 us; error 38 us; skew 500 ppm
Sep 24 21:16:23 dist-test-slave-q276 krb5kdc[27942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284583, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:23.654510 27955 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 21:16:23.655035 27955 webserver.cc:460] Webserver started at http://127.0.0.1:37949/ using document root <none> and password file <none>
I20260924 21:16:23.655288 27955 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:23.655359 27955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:23.655540 27955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:23.657003 27955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "f1582dab6f284c519d8a9b25d2ec0b59"
format_stamp: "Formatted at 2026-09-24 21:16:23 on dist-test-slave-q276"
server_key: "5577ec26c9eb85d5bbe443678711cffb"
server_key_iv: "f9863c7185e2e341d36ea580420c980e"
server_key_version: "encryptionkey@0"
I20260924 21:16:23.657465 27955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "f1582dab6f284c519d8a9b25d2ec0b59"
format_stamp: "Formatted at 2026-09-24 21:16:23 on dist-test-slave-q276"
server_key: "5577ec26c9eb85d5bbe443678711cffb"
server_key_iv: "f9863c7185e2e341d36ea580420c980e"
server_key_version: "encryptionkey@0"
I20260924 21:16:23.659045 27955 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:16:23.659982 27970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:23.660173 27955 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:23.660290 27955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "f1582dab6f284c519d8a9b25d2ec0b59"
format_stamp: "Formatted at 2026-09-24 21:16:23 on dist-test-slave-q276"
server_key: "5577ec26c9eb85d5bbe443678711cffb"
server_key_iv: "f9863c7185e2e341d36ea580420c980e"
server_key_version: "encryptionkey@0"
I20260924 21:16:23.660362 27955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:23.673908 27955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:23.676384 27955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:23.676554 27955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:23.681413 27955 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41265
I20260924 21:16:23.681468 28022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41265 every 8 connection(s)
I20260924 21:16:23.681823 27955 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:16:23.682641 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 27955
I20260924 21:16:23.682750 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:16:23.682991 6986 external_mini_cluster.cc:1428] Setting key 7f5dc60ce3c1afff91ce694dad3be5d1
I20260924 21:16:23.683926 28023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:23.687606 28023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1582dab6f284c519d8a9b25d2ec0b59: Bootstrap starting.
I20260924 21:16:23.688257 28023 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1582dab6f284c519d8a9b25d2ec0b59: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:23.688511 28026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:23.683615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:45545 (local address 127.0.0.1:41265)
0924 21:16:23.683810 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:23.683819 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:23.683851 (+ 32us) server_negotiation.cc:407] Connection header received
0924 21:16:23.684339 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:23.684347 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:23.684398 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:23.684555 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:23.684761 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:23.685651 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:23.686570 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:23.686981 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:23.687102 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:23.687860 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:23.687864 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:23.687864 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:23.688043 (+ 179us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:23.688073 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:23.688083 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:23.688175 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:23.688259 (+ 84us) server_negotiation.cc:299] Negotiation successful
0924 21:16:23.688288 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":91,"threads_started":1}
I20260924 21:16:23.688630 28023 log.cc:824] T 00000000000000000000000000000000 P f1582dab6f284c519d8a9b25d2ec0b59: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:23.689349 28023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1582dab6f284c519d8a9b25d2ec0b59: No bootstrap required, opened a new log
I20260924 21:16:23.689491 6986 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:16:23.691974 28023 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1582dab6f284c519d8a9b25d2ec0b59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1582dab6f284c519d8a9b25d2ec0b59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41265 } }
I20260924 21:16:23.692149 28023 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1582dab6f284c519d8a9b25d2ec0b59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:23.692193 28023 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1582dab6f284c519d8a9b25d2ec0b59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1582dab6f284c519d8a9b25d2ec0b59, State: Initialized, Role: FOLLOWER
I20260924 21:16:23.692317 28023 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1582dab6f284c519d8a9b25d2ec0b59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1582dab6f284c519d8a9b25d2ec0b59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41265 } }
I20260924 21:16:23.692423 28023 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1582dab6f284c519d8a9b25d2ec0b59 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:16:23.692494 28023 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1582dab6f284c519d8a9b25d2ec0b59 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:16:23.692546 28023 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1582dab6f284c519d8a9b25d2ec0b59 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:23.693173 28023 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1582dab6f284c519d8a9b25d2ec0b59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1582dab6f284c519d8a9b25d2ec0b59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41265 } }
I20260924 21:16:23.693284 28023 leader_election.cc:302] T 00000000000000000000000000000000 P f1582dab6f284c519d8a9b25d2ec0b59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1582dab6f284c519d8a9b25d2ec0b59; no voters:
I20260924 21:16:23.693468 28023 leader_election.cc:288] T 00000000000000000000000000000000 P f1582dab6f284c519d8a9b25d2ec0b59 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:16:23.693509 28033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1582dab6f284c519d8a9b25d2ec0b59 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:23.693645 28033 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1582dab6f284c519d8a9b25d2ec0b59 [term 1 LEADER]: Becoming Leader. State: Replica: f1582dab6f284c519d8a9b25d2ec0b59, State: Running, Role: LEADER
I20260924 21:16:23.693744 28023 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1582dab6f284c519d8a9b25d2ec0b59 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:23.693746 28033 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1582dab6f284c519d8a9b25d2ec0b59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1582dab6f284c519d8a9b25d2ec0b59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41265 } }
I20260924 21:16:23.694352 28035 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1582dab6f284c519d8a9b25d2ec0b59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1582dab6f284c519d8a9b25d2ec0b59. Latest consensus state: current_term: 1 leader_uuid: "f1582dab6f284c519d8a9b25d2ec0b59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1582dab6f284c519d8a9b25d2ec0b59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41265 } } }
I20260924 21:16:23.694494 28035 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1582dab6f284c519d8a9b25d2ec0b59 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:23.694192 28034 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1582dab6f284c519d8a9b25d2ec0b59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1582dab6f284c519d8a9b25d2ec0b59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1582dab6f284c519d8a9b25d2ec0b59" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41265 } } }
I20260924 21:16:23.694761 28034 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1582dab6f284c519d8a9b25d2ec0b59 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:23.694881 28042 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:16:23.695580 28026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:23.691043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:49835 (local address 127.0.0.1:41265)
0924 21:16:23.691101 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:23.691103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:23.691171 (+ 68us) server_negotiation.cc:407] Connection header received
0924 21:16:23.691220 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:23.691221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:23.691232 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:23.691316 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:23.691408 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:23.692104 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:23.693004 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:23.693288 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:23.693359 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:23.695149 (+ 1790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:23.695151 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:23.695152 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:23.695156 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:23.695172 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:23.695177 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:23.695237 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:23.695497 (+ 260us) server_negotiation.cc:299] Negotiation successful
0924 21:16:23.695509 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 21:16:23.695619 28042 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:16:23.697037 28042 catalog_manager.cc:1347] Generated new cluster ID: 977fbc77dfca4488ab5c9da939f89e9b
I20260924 21:16:23.697098 28042 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:16:23.732597 28042 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:16:23.733259 28042 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:16:23.738747 28042 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1582dab6f284c519d8a9b25d2ec0b59: Generated new TSK 0
I20260924 21:16:23.739013 28042 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:16:23.747087 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 27955
W20260924 21:16:23.753714 27940 connection.cc:535] client connection to 127.0.0.1:41265 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41265): Connection reset by peer (error 104)
2026-09-24T21:16:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1366 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:16:23 dist-test-slave-q276 krb5kdc[28054](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:16:23 dist-test-slave-q276 krb5kdc[28054](info): set up 2 sockets
Sep 24 21:16:23 dist-test-slave-q276 krb5kdc[28054](info): commencing operation
krb5kdc: starting...
W20260924 21:16:24.786381 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:16:24 dist-test-slave-q276 krb5kdc[28054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:16:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:16:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 21:16:24.906070 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43521
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:24.989552 28067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:24.989777 28067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:24.989801 28067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:24.991153 28067 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:24.991199 28067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:24.991216 28067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:24.991230 28067 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:24.991245 28067 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:24.992789 28067 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:24.992988 28067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:24.993172 28067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:24.998370 28075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:24.998392 28073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:24.998461 28067 server_base.cc:1034] running on GCE node
W20260924 21:16:24.998389 28072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:24.998819 28067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:24.999128 28067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:25.000313 28067 hybrid_clock.cc:648] HybridClock initialized: now 1790284585000290 us; error 34 us; skew 500 ppm
Sep 24 21:16:24 dist-test-slave-q276 krb5kdc[28054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284584, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:25.006527 28067 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 21:16:25.007071 28067 webserver.cc:460] Webserver started at http://127.0.0.1:38765/ using document root <none> and password file <none>
I20260924 21:16:25.007334 28067 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:25.007406 28067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:25.007570 28067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:25.008685 28067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "a649c08f7639404794f5dbe22fcf274f"
format_stamp: "Formatted at 2026-09-24 21:16:25 on dist-test-slave-q276"
server_key: "81d247dea09cb51a07858b42c6d40bbd"
server_key_iv: "72f6080f7e3c5b4b68f50e89b3600405"
server_key_version: "encryptionkey@0"
I20260924 21:16:25.009063 28067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "a649c08f7639404794f5dbe22fcf274f"
format_stamp: "Formatted at 2026-09-24 21:16:25 on dist-test-slave-q276"
server_key: "81d247dea09cb51a07858b42c6d40bbd"
server_key_iv: "72f6080f7e3c5b4b68f50e89b3600405"
server_key_version: "encryptionkey@0"
I20260924 21:16:25.010584 28067 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:16:25.011379 28082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:25.011581 28067 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:16:25.011694 28067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "a649c08f7639404794f5dbe22fcf274f"
format_stamp: "Formatted at 2026-09-24 21:16:25 on dist-test-slave-q276"
server_key: "81d247dea09cb51a07858b42c6d40bbd"
server_key_iv: "72f6080f7e3c5b4b68f50e89b3600405"
server_key_version: "encryptionkey@0"
I20260924 21:16:25.011770 28067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:25.015210 28067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:25.017170 28067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:25.017338 28067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:25.021354 28067 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43521
I20260924 21:16:25.021387 28134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43521 every 8 connection(s)
I20260924 21:16:25.021741 28067 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:16:25.022866 28135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:25.025267 28135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a649c08f7639404794f5dbe22fcf274f: Bootstrap starting.
I20260924 21:16:25.025925 28135 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a649c08f7639404794f5dbe22fcf274f: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:25.026240 28135 log.cc:824] T 00000000000000000000000000000000 P a649c08f7639404794f5dbe22fcf274f: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:25.027099 28135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a649c08f7639404794f5dbe22fcf274f: No bootstrap required, opened a new log
I20260924 21:16:25.029313 28135 raft_consensus.cc:348] T 00000000000000000000000000000000 P a649c08f7639404794f5dbe22fcf274f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a649c08f7639404794f5dbe22fcf274f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43521 } }
I20260924 21:16:25.029459 28135 raft_consensus.cc:374] T 00000000000000000000000000000000 P a649c08f7639404794f5dbe22fcf274f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:25.029484 28135 raft_consensus.cc:729] T 00000000000000000000000000000000 P a649c08f7639404794f5dbe22fcf274f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a649c08f7639404794f5dbe22fcf274f, State: Initialized, Role: FOLLOWER
I20260924 21:16:25.029663 28135 consensus_queue.cc:260] T 00000000000000000000000000000000 P a649c08f7639404794f5dbe22fcf274f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a649c08f7639404794f5dbe22fcf274f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43521 } }
I20260924 21:16:25.029727 28135 raft_consensus.cc:388] T 00000000000000000000000000000000 P a649c08f7639404794f5dbe22fcf274f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:16:25.029788 28135 raft_consensus.cc:482] T 00000000000000000000000000000000 P a649c08f7639404794f5dbe22fcf274f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:16:25.029863 28135 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a649c08f7639404794f5dbe22fcf274f [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:25.030529 28135 raft_consensus.cc:504] T 00000000000000000000000000000000 P a649c08f7639404794f5dbe22fcf274f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a649c08f7639404794f5dbe22fcf274f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43521 } }
I20260924 21:16:25.030648 28135 leader_election.cc:302] T 00000000000000000000000000000000 P a649c08f7639404794f5dbe22fcf274f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a649c08f7639404794f5dbe22fcf274f; no voters:
I20260924 21:16:25.030840 28135 leader_election.cc:288] T 00000000000000000000000000000000 P a649c08f7639404794f5dbe22fcf274f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:16:25.030908 28138 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a649c08f7639404794f5dbe22fcf274f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:25.031126 28135 sys_catalog.cc:564] T 00000000000000000000000000000000 P a649c08f7639404794f5dbe22fcf274f [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:25.031181 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 28067
I20260924 21:16:25.031298 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:16:25.031498 6986 external_mini_cluster.cc:1428] Setting key abf86df48ab69f302dafa168ecfe2197
I20260924 21:16:25.031808 28138 raft_consensus.cc:686] T 00000000000000000000000000000000 P a649c08f7639404794f5dbe22fcf274f [term 1 LEADER]: Becoming Leader. State: Replica: a649c08f7639404794f5dbe22fcf274f, State: Running, Role: LEADER
I20260924 21:16:25.031908 28138 consensus_queue.cc:237] T 00000000000000000000000000000000 P a649c08f7639404794f5dbe22fcf274f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a649c08f7639404794f5dbe22fcf274f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43521 } }
I20260924 21:16:25.032277 28138 sys_catalog.cc:455] T 00000000000000000000000000000000 P a649c08f7639404794f5dbe22fcf274f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a649c08f7639404794f5dbe22fcf274f. Latest consensus state: current_term: 1 leader_uuid: "a649c08f7639404794f5dbe22fcf274f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a649c08f7639404794f5dbe22fcf274f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43521 } } }
I20260924 21:16:25.032284 28139 sys_catalog.cc:455] T 00000000000000000000000000000000 P a649c08f7639404794f5dbe22fcf274f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a649c08f7639404794f5dbe22fcf274f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a649c08f7639404794f5dbe22fcf274f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43521 } } }
I20260924 21:16:25.032364 28138 sys_catalog.cc:458] T 00000000000000000000000000000000 P a649c08f7639404794f5dbe22fcf274f [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:25.032517 28139 sys_catalog.cc:458] T 00000000000000000000000000000000 P a649c08f7639404794f5dbe22fcf274f [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:25.032888 28144 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:16:25.033449 28144 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:16:25.034564 28142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:25.031990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:39637 (local address 127.0.0.1:43521)
0924 21:16:25.032146 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:25.032156 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:25.032702 (+ 546us) server_negotiation.cc:407] Connection header received
0924 21:16:25.032839 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:25.032847 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:25.032882 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:25.033028 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:25.034122 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:25.034126 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:25.034127 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:25.034132 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:25.034178 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:25.034192 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:25.034294 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:25.034360 (+ 66us) server_negotiation.cc:299] Negotiation successful
0924 21:16:25.034414 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":69,"threads_started":1}
I20260924 21:16:25.035422 6986 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:16:25.035616 28144 catalog_manager.cc:1347] Generated new cluster ID: 318065d9827247d09a00a9a83f6d79d6
I20260924 21:16:25.035669 28144 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:16:25.037935 28142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:25.036848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:58629 (local address 127.0.0.1:43521)
0924 21:16:25.036882 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:25.036884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:25.037008 (+ 124us) server_negotiation.cc:407] Connection header received
0924 21:16:25.037083 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:25.037084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:25.037089 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:25.037145 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:25.037708 (+ 563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:25.037710 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:25.037710 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:25.037714 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:25.037727 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:25.037731 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:25.037786 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:25.037842 (+ 56us) server_negotiation.cc:299] Negotiation successful
0924 21:16:25.037856 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 21:16:25.043952 28144 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:16:25.044118 28144 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:16:25.054351 28144 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a649c08f7639404794f5dbe22fcf274f: Generated new TSK 0
I20260924 21:16:25.054592 28144 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:16:25.057626 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 28067
2026-09-24T21:16:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1308 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:16:25 dist-test-slave-q276 krb5kdc[28166](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:16:25 dist-test-slave-q276 krb5kdc[28166](info): set up 2 sockets
Sep 24 21:16:25 dist-test-slave-q276 krb5kdc[28166](info): commencing operation
krb5kdc: starting...
W20260924 21:16:26.094444 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:16:26 dist-test-slave-q276 krb5kdc[28166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:16:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:16:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 21:16:26.209347 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40241
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:26.289628 28179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:26.289837 28179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:26.289860 28179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:26.291114 28179 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:26.291157 28179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:26.291172 28179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:26.291209 28179 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:26.291236 28179 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:26.292695 28179 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40241
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:26.292948 28179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:26.293175 28179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:26.298041 28185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:26.298049 28184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:26.298229 28179 server_base.cc:1034] running on GCE node
W20260924 21:16:26.298297 28187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:26.298557 28179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:26.298856 28179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:26.300017 28179 hybrid_clock.cc:648] HybridClock initialized: now 1790284586300014 us; error 37 us; skew 500 ppm
Sep 24 21:16:26 dist-test-slave-q276 krb5kdc[28166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284586, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:26.302486 28179 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 21:16:26.302903 28179 webserver.cc:460] Webserver started at http://127.0.0.1:40523/ using document root <none> and password file <none>
I20260924 21:16:26.303081 28179 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:26.303120 28179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:26.303207 28179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:26.304210 28179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "f0542dc46c1543e1a65d7d4eddaade94"
format_stamp: "Formatted at 2026-09-24 21:16:26 on dist-test-slave-q276"
server_key: "bd53d72e6cdf3d71e763711827a34a39"
server_key_iv: "55c730bf97d1d0b2e802805e76fe2d6f"
server_key_version: "encryptionkey@0"
I20260924 21:16:26.304497 28179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "f0542dc46c1543e1a65d7d4eddaade94"
format_stamp: "Formatted at 2026-09-24 21:16:26 on dist-test-slave-q276"
server_key: "bd53d72e6cdf3d71e763711827a34a39"
server_key_iv: "55c730bf97d1d0b2e802805e76fe2d6f"
server_key_version: "encryptionkey@0"
I20260924 21:16:26.305735 28179 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:16:26.306432 28194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:26.306622 28179 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 21:16:26.306708 28179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "f0542dc46c1543e1a65d7d4eddaade94"
format_stamp: "Formatted at 2026-09-24 21:16:26 on dist-test-slave-q276"
server_key: "bd53d72e6cdf3d71e763711827a34a39"
server_key_iv: "55c730bf97d1d0b2e802805e76fe2d6f"
server_key_version: "encryptionkey@0"
I20260924 21:16:26.306761 28179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:26.314684 28179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:26.316919 28179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:26.317027 28179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:26.320905 28179 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40241
I20260924 21:16:26.320936 28246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40241 every 8 connection(s)
I20260924 21:16:26.321302 28179 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:16:26.322252 28247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:26.323829 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 28179
I20260924 21:16:26.323923 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:16:26.324134 6986 external_mini_cluster.cc:1428] Setting key 9779fd0446f5175bcd495b320d896013
I20260924 21:16:26.325065 28247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0542dc46c1543e1a65d7d4eddaade94: Bootstrap starting.
I20260924 21:16:26.325757 28247 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0542dc46c1543e1a65d7d4eddaade94: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:26.326068 28247 log.cc:824] T 00000000000000000000000000000000 P f0542dc46c1543e1a65d7d4eddaade94: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:26.326813 28247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0542dc46c1543e1a65d7d4eddaade94: No bootstrap required, opened a new log
I20260924 21:16:26.328958 28247 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0542dc46c1543e1a65d7d4eddaade94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0542dc46c1543e1a65d7d4eddaade94" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40241 } }
I20260924 21:16:26.329169 28247 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0542dc46c1543e1a65d7d4eddaade94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:26.329211 28247 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0542dc46c1543e1a65d7d4eddaade94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0542dc46c1543e1a65d7d4eddaade94, State: Initialized, Role: FOLLOWER
I20260924 21:16:26.329375 28247 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0542dc46c1543e1a65d7d4eddaade94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0542dc46c1543e1a65d7d4eddaade94" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40241 } }
I20260924 21:16:26.329474 28247 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0542dc46c1543e1a65d7d4eddaade94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:16:26.329523 28247 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0542dc46c1543e1a65d7d4eddaade94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:16:26.329568 28247 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0542dc46c1543e1a65d7d4eddaade94 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:26.330392 28247 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0542dc46c1543e1a65d7d4eddaade94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0542dc46c1543e1a65d7d4eddaade94" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40241 } }
I20260924 21:16:26.330500 28247 leader_election.cc:302] T 00000000000000000000000000000000 P f0542dc46c1543e1a65d7d4eddaade94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0542dc46c1543e1a65d7d4eddaade94; no voters:
I20260924 21:16:26.330668 28247 leader_election.cc:288] T 00000000000000000000000000000000 P f0542dc46c1543e1a65d7d4eddaade94 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:16:26.330721 28252 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0542dc46c1543e1a65d7d4eddaade94 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:26.330873 28252 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0542dc46c1543e1a65d7d4eddaade94 [term 1 LEADER]: Becoming Leader. State: Replica: f0542dc46c1543e1a65d7d4eddaade94, State: Running, Role: LEADER
I20260924 21:16:26.330963 28247 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0542dc46c1543e1a65d7d4eddaade94 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:26.331012 28252 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0542dc46c1543e1a65d7d4eddaade94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0542dc46c1543e1a65d7d4eddaade94" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40241 } }
Sep 24 21:16:26 dist-test-slave-q276 krb5kdc[28166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260924 21:16:26.331480 28254 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0542dc46c1543e1a65d7d4eddaade94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0542dc46c1543e1a65d7d4eddaade94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0542dc46c1543e1a65d7d4eddaade94" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40241 } } }
I20260924 21:16:26.331683 28254 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0542dc46c1543e1a65d7d4eddaade94 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:26.331619 28252 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0542dc46c1543e1a65d7d4eddaade94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0542dc46c1543e1a65d7d4eddaade94. Latest consensus state: current_term: 1 leader_uuid: "f0542dc46c1543e1a65d7d4eddaade94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0542dc46c1543e1a65d7d4eddaade94" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40241 } } }
I20260924 21:16:26.331936 28252 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0542dc46c1543e1a65d7d4eddaade94 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:26.332262 28261 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:16:26.333176 28261 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:16:26.334861 28261 catalog_manager.cc:1347] Generated new cluster ID: 6c1635af80a44d63884e78d238577f62
I20260924 21:16:26.334913 28261 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:16:26.336236 28250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:26.324698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:48237 (local address 127.0.0.1:40241)
0924 21:16:26.324886 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:26.324895 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:26.324927 (+ 32us) server_negotiation.cc:407] Connection header received
0924 21:16:26.325500 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:26.325508 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:26.325572 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:26.325729 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:26.326433 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:26.327422 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:26.328541 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:26.328929 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:26.329023 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:26.331987 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:26.331991 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:26.331992 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:26.332003 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:26.334131 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:26.334618 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:26.334621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:26.334621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:26.334732 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:26.335007 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:26.335008 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:26.335008 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:26.335314 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:26.335464 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:26.335909 (+ 445us) server_negotiation.cc:299] Negotiation successful
0924 21:16:26.336009 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":82,"threads_started":1}
I20260924 21:16:26.337172 6986 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:16:26.345620 28250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:26.338509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:45435 (local address 127.0.0.1:40241)
0924 21:16:26.338543 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:26.338545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:26.338710 (+ 165us) server_negotiation.cc:407] Connection header received
0924 21:16:26.338732 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:26.338733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:26.338743 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:26.338796 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:26.339398 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:26.340199 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:26.341137 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:26.341463 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:26.341537 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:26.342829 (+ 1292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:26.342831 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:26.342831 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:26.342838 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:26.344403 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:26.344786 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:26.344787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:26.344787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:26.344845 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:26.345083 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:26.345084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:26.345084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:26.345216 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:26.345321 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:26.345450 (+ 129us) server_negotiation.cc:299] Negotiation successful
0924 21:16:26.345512 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 21:16:26.367781 28261 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:16:26.368525 28261 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:16:26.378705 28261 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0542dc46c1543e1a65d7d4eddaade94: Generated new TSK 0
I20260924 21:16:26.378934 28261 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:16:26.380985 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 28179
W20260924 21:16:26.385351 28164 connection.cc:535] client connection to 127.0.0.1:40241 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40241): Connection reset by peer (error 104)
2026-09-24T21:16:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1323 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:16:26 dist-test-slave-q276 krb5kdc[28278](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:16:26 dist-test-slave-q276 krb5kdc[28278](info): set up 2 sockets
Sep 24 21:16:26 dist-test-slave-q276 krb5kdc[28278](info): commencing operation
krb5kdc: starting...
W20260924 21:16:27.414546 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:16:27 dist-test-slave-q276 krb5kdc[28278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:16:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:16:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 21:16:27.536451 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42045
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42045
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:27.619740 28291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:27.620025 28291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:27.620074 28291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:27.621430 28291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:27.621518 28291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:27.621548 28291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:27.621585 28291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:27.621611 28291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:27.623315 28291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42045
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:27.623637 28291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:27.623888 28291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:27.629208 28296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:27.629221 28299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:27.629307 28297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:27.629480 28291 server_base.cc:1034] running on GCE node
I20260924 21:16:27.629770 28291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:27.630066 28291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:27.631225 28291 hybrid_clock.cc:648] HybridClock initialized: now 1790284587631203 us; error 39 us; skew 500 ppm
Sep 24 21:16:27 dist-test-slave-q276 krb5kdc[28278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284587, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:27.633785 28291 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 21:16:27.634235 28291 webserver.cc:460] Webserver started at http://127.0.0.1:35897/ using document root <none> and password file <none>
I20260924 21:16:27.634505 28291 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:27.634573 28291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:27.634737 28291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:27.635820 28291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "8a12e815ad81481ebe1f7ea46487ca40"
format_stamp: "Formatted at 2026-09-24 21:16:27 on dist-test-slave-q276"
server_key: "f5fe4c814a2f02caa4e57873e709ddab"
server_key_iv: "0954dc1235004c2971007ceb8b248fab"
server_key_version: "encryptionkey@0"
I20260924 21:16:27.636195 28291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "8a12e815ad81481ebe1f7ea46487ca40"
format_stamp: "Formatted at 2026-09-24 21:16:27 on dist-test-slave-q276"
server_key: "f5fe4c814a2f02caa4e57873e709ddab"
server_key_iv: "0954dc1235004c2971007ceb8b248fab"
server_key_version: "encryptionkey@0"
I20260924 21:16:27.637527 28291 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:16:27.638363 28306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:27.638561 28291 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:27.638667 28291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "8a12e815ad81481ebe1f7ea46487ca40"
format_stamp: "Formatted at 2026-09-24 21:16:27 on dist-test-slave-q276"
server_key: "f5fe4c814a2f02caa4e57873e709ddab"
server_key_iv: "0954dc1235004c2971007ceb8b248fab"
server_key_version: "encryptionkey@0"
I20260924 21:16:27.638752 28291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:27.664824 28291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:27.667358 28291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:27.667533 28291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:27.671716 28291 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42045
I20260924 21:16:27.671804 28358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42045 every 8 connection(s)
I20260924 21:16:27.672173 28291 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:16:27.673075 28359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:27.675462 28359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a12e815ad81481ebe1f7ea46487ca40: Bootstrap starting.
I20260924 21:16:27.676071 28359 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a12e815ad81481ebe1f7ea46487ca40: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:27.676352 28359 log.cc:824] T 00000000000000000000000000000000 P 8a12e815ad81481ebe1f7ea46487ca40: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:27.677042 28359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a12e815ad81481ebe1f7ea46487ca40: No bootstrap required, opened a new log
I20260924 21:16:27.678711 28359 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a12e815ad81481ebe1f7ea46487ca40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a12e815ad81481ebe1f7ea46487ca40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42045 } }
I20260924 21:16:27.678854 28359 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a12e815ad81481ebe1f7ea46487ca40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:27.678877 28359 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a12e815ad81481ebe1f7ea46487ca40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a12e815ad81481ebe1f7ea46487ca40, State: Initialized, Role: FOLLOWER
I20260924 21:16:27.679070 28359 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a12e815ad81481ebe1f7ea46487ca40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a12e815ad81481ebe1f7ea46487ca40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42045 } }
I20260924 21:16:27.679157 28359 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a12e815ad81481ebe1f7ea46487ca40 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:16:27.679211 28359 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a12e815ad81481ebe1f7ea46487ca40 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:16:27.679265 28359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a12e815ad81481ebe1f7ea46487ca40 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:27.679821 28359 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a12e815ad81481ebe1f7ea46487ca40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a12e815ad81481ebe1f7ea46487ca40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42045 } }
I20260924 21:16:27.679908 28359 leader_election.cc:302] T 00000000000000000000000000000000 P 8a12e815ad81481ebe1f7ea46487ca40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a12e815ad81481ebe1f7ea46487ca40; no voters:
I20260924 21:16:27.680127 28359 leader_election.cc:288] T 00000000000000000000000000000000 P 8a12e815ad81481ebe1f7ea46487ca40 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:16:27.680199 28362 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a12e815ad81481ebe1f7ea46487ca40 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:27.680368 28362 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a12e815ad81481ebe1f7ea46487ca40 [term 1 LEADER]: Becoming Leader. State: Replica: 8a12e815ad81481ebe1f7ea46487ca40, State: Running, Role: LEADER
I20260924 21:16:27.680374 28359 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a12e815ad81481ebe1f7ea46487ca40 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:27.680469 28362 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a12e815ad81481ebe1f7ea46487ca40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a12e815ad81481ebe1f7ea46487ca40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42045 } }
I20260924 21:16:27.680866 28364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a12e815ad81481ebe1f7ea46487ca40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a12e815ad81481ebe1f7ea46487ca40. Latest consensus state: current_term: 1 leader_uuid: "8a12e815ad81481ebe1f7ea46487ca40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a12e815ad81481ebe1f7ea46487ca40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42045 } } }
I20260924 21:16:27.681007 28363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a12e815ad81481ebe1f7ea46487ca40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a12e815ad81481ebe1f7ea46487ca40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a12e815ad81481ebe1f7ea46487ca40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42045 } } }
I20260924 21:16:27.681007 28364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a12e815ad81481ebe1f7ea46487ca40 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:27.681123 28363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a12e815ad81481ebe1f7ea46487ca40 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:27.681388 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 28291
I20260924 21:16:27.681501 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:16:27.681690 6986 external_mini_cluster.cc:1428] Setting key dfd466ab600528e08ecf5259cd23f781
W20260924 21:16:27.682900 28379 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8a12e815ad81481ebe1f7ea46487ca40: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 21:16:27.682965 28379 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 21:16:27.683074 28380 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:16:27.683275 28380 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:16:27.687852 28376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:27.682149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:54897 (local address 127.0.0.1:42045)
0924 21:16:27.682366 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:27.682377 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:27.682424 (+ 47us) server_negotiation.cc:407] Connection header received
0924 21:16:27.682611 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:27.682619 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:27.682692 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:27.682866 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:27.683106 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:27.684643 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:27.685792 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:27.686238 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:27.686422 (+ 184us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:27.687133 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:27.687137 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:27.687138 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:27.687325 (+ 187us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:27.687356 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:27.687368 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:27.687436 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:27.687551 (+ 115us) server_negotiation.cc:299] Negotiation successful
0924 21:16:27.687595 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":80,"threads_started":1}
I20260924 21:16:27.688823 6986 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:16:27.693979 28376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:27.690482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:39825 (local address 127.0.0.1:42045)
0924 21:16:27.690540 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:27.690542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:27.690698 (+ 156us) server_negotiation.cc:407] Connection header received
0924 21:16:27.690746 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:27.690747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:27.690757 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:27.690807 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:27.690953 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:27.691489 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:27.692661 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:27.692874 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:27.692942 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:27.693651 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:27.693653 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:27.693653 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:27.693657 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:27.693672 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:27.693676 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:27.693728 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:27.693828 (+ 100us) server_negotiation.cc:299] Negotiation successful
0924 21:16:27.693840 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 21:16:27.694671 28380 catalog_manager.cc:1347] Generated new cluster ID: aae3b54be3e647c3bd8ca501609519d5
I20260924 21:16:27.694739 28380 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:16:27.718959 28380 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:16:27.719744 28380 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:16:27.726190 28380 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a12e815ad81481ebe1f7ea46487ca40: Generated new TSK 0
I20260924 21:16:27.726409 28380 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:16:27.728848 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 28291
W20260924 21:16:27.733196 28276 connection.cc:535] client connection to 127.0.0.1:42045 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42045): Connection reset by peer (error 104)
2026-09-24T21:16:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1347 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:16:27 dist-test-slave-q276 krb5kdc[28390](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:16:27 dist-test-slave-q276 krb5kdc[28390](info): set up 2 sockets
Sep 24 21:16:27 dist-test-slave-q276 krb5kdc[28390](info): commencing operation
krb5kdc: starting...
W20260924 21:16:28.770398 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:16:28 dist-test-slave-q276 krb5kdc[28390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:16:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:16:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 21:16:28.887346 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44401
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44401
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:28.968520 28403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:28.968732 28403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:28.968755 28403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:28.969915 28403 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:28.969960 28403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:28.969975 28403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:28.969987 28403 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:28.970042 28403 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:28.971517 28403 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44401
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284333163253-6986-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:28.971833 28403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:28.972064 28403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 21:16:28.977468 28403 server_base.cc:1034] running on GCE node
W20260924 21:16:28.977432 28411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:28.977480 28408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:28.977450 28409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:28.977900 28403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:28.978204 28403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:28.979372 28403 hybrid_clock.cc:648] HybridClock initialized: now 1790284588979350 us; error 39 us; skew 500 ppm
Sep 24 21:16:28 dist-test-slave-q276 krb5kdc[28390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284588, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:28.982014 28403 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 21:16:28.982519 28403 webserver.cc:460] Webserver started at http://127.0.0.1:40759/ using document root <none> and password file <none>
I20260924 21:16:28.982771 28403 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:28.982841 28403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:28.982982 28403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:28.984021 28403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "ffcf3a6c4cef466da32c55eb60acf4fb"
format_stamp: "Formatted at 2026-09-24 21:16:28 on dist-test-slave-q276"
server_key: "9443f7e46b89d186ac1e59a33b52766a"
server_key_iv: "27796b102b08f20f389cd3ea3e69f331"
server_key_version: "encryptionkey@0"
I20260924 21:16:28.984380 28403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "ffcf3a6c4cef466da32c55eb60acf4fb"
format_stamp: "Formatted at 2026-09-24 21:16:28 on dist-test-slave-q276"
server_key: "9443f7e46b89d186ac1e59a33b52766a"
server_key_iv: "27796b102b08f20f389cd3ea3e69f331"
server_key_version: "encryptionkey@0"
I20260924 21:16:28.985682 28403 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:16:28.986465 28418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:28.986729 28403 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:28.986836 28403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "ffcf3a6c4cef466da32c55eb60acf4fb"
format_stamp: "Formatted at 2026-09-24 21:16:28 on dist-test-slave-q276"
server_key: "9443f7e46b89d186ac1e59a33b52766a"
server_key_iv: "27796b102b08f20f389cd3ea3e69f331"
server_key_version: "encryptionkey@0"
I20260924 21:16:28.986912 28403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:28.990159 28403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:28.992040 28403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:28.992193 28403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:28.996062 28403 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44401
I20260924 21:16:28.996091 28470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44401 every 8 connection(s)
I20260924 21:16:28.996408 28403 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:16:28.997334 28471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:28.999598 28471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffcf3a6c4cef466da32c55eb60acf4fb: Bootstrap starting.
I20260924 21:16:29.000098 28471 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffcf3a6c4cef466da32c55eb60acf4fb: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:29.000347 28471 log.cc:824] T 00000000000000000000000000000000 P ffcf3a6c4cef466da32c55eb60acf4fb: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:29.000978 28471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffcf3a6c4cef466da32c55eb60acf4fb: No bootstrap required, opened a new log
I20260924 21:16:29.002153 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 28403
I20260924 21:16:29.002270 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:16:29.002456 6986 external_mini_cluster.cc:1428] Setting key be69ddce41a3fbac8634738911785c40
I20260924 21:16:29.002735 28471 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffcf3a6c4cef466da32c55eb60acf4fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffcf3a6c4cef466da32c55eb60acf4fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44401 } }
I20260924 21:16:29.002942 28471 raft_consensus.cc:374] T 00000000000000000000000000000000 P ffcf3a6c4cef466da32c55eb60acf4fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:29.003026 28471 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffcf3a6c4cef466da32c55eb60acf4fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffcf3a6c4cef466da32c55eb60acf4fb, State: Initialized, Role: FOLLOWER
I20260924 21:16:29.003172 28471 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffcf3a6c4cef466da32c55eb60acf4fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffcf3a6c4cef466da32c55eb60acf4fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44401 } }
I20260924 21:16:29.003262 28471 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffcf3a6c4cef466da32c55eb60acf4fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:16:29.003296 28471 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffcf3a6c4cef466da32c55eb60acf4fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:16:29.003348 28471 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffcf3a6c4cef466da32c55eb60acf4fb [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:29.004544 28471 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffcf3a6c4cef466da32c55eb60acf4fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffcf3a6c4cef466da32c55eb60acf4fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44401 } }
I20260924 21:16:29.004655 28471 leader_election.cc:302] T 00000000000000000000000000000000 P ffcf3a6c4cef466da32c55eb60acf4fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffcf3a6c4cef466da32c55eb60acf4fb; no voters:
I20260924 21:16:29.004777 28475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:29.002986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:48601 (local address 127.0.0.1:44401)
0924 21:16:29.003181 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:29.003190 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:29.003250 (+ 60us) server_negotiation.cc:407] Connection header received
0924 21:16:29.003389 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:29.003397 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:29.003427 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:29.003543 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:29.004205 (+ 662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:29.004209 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:29.004209 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:29.004212 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:29.004249 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:29.004260 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:29.004314 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:29.004461 (+ 147us) server_negotiation.cc:299] Negotiation successful
0924 21:16:29.004514 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":81,"threads_started":1}
I20260924 21:16:29.004814 28471 leader_election.cc:288] T 00000000000000000000000000000000 P ffcf3a6c4cef466da32c55eb60acf4fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:16:29.004844 28476 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffcf3a6c4cef466da32c55eb60acf4fb [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:29.004945 28476 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffcf3a6c4cef466da32c55eb60acf4fb [term 1 LEADER]: Becoming Leader. State: Replica: ffcf3a6c4cef466da32c55eb60acf4fb, State: Running, Role: LEADER
I20260924 21:16:29.005052 28471 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffcf3a6c4cef466da32c55eb60acf4fb [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:29.005261 28476 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffcf3a6c4cef466da32c55eb60acf4fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffcf3a6c4cef466da32c55eb60acf4fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44401 } }
I20260924 21:16:29.005710 6986 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:16:29.006948 28478 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffcf3a6c4cef466da32c55eb60acf4fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffcf3a6c4cef466da32c55eb60acf4fb. Latest consensus state: current_term: 1 leader_uuid: "ffcf3a6c4cef466da32c55eb60acf4fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffcf3a6c4cef466da32c55eb60acf4fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44401 } } }
I20260924 21:16:29.007057 28478 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffcf3a6c4cef466da32c55eb60acf4fb [sys.catalog]: This master's current role is: LEADER
W20260924 21:16:29.007720 28491 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ffcf3a6c4cef466da32c55eb60acf4fb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 21:16:29.007781 28491 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 21:16:29.007838 28496 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:16:29.007998 28496 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:16:29.008478 28475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:29.007326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.215:49517 (local address 127.0.0.1:44401)
0924 21:16:29.007362 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:29.007364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:29.007466 (+ 102us) server_negotiation.cc:407] Connection header received
0924 21:16:29.007538 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:29.007539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:29.007545 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:29.007601 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:29.008226 (+ 625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:29.008228 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:29.008228 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:29.008232 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:29.008246 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:29.008250 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:29.008326 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:29.008381 (+ 55us) server_negotiation.cc:299] Negotiation successful
0924 21:16:29.008394 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 21:16:29.009253 28477 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffcf3a6c4cef466da32c55eb60acf4fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ffcf3a6c4cef466da32c55eb60acf4fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffcf3a6c4cef466da32c55eb60acf4fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44401 } } }
I20260924 21:16:29.009327 28477 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffcf3a6c4cef466da32c55eb60acf4fb [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:29.009572 28496 catalog_manager.cc:1347] Generated new cluster ID: 77394d212ec4473797a7165119cd7bd5
I20260924 21:16:29.009618 28496 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:16:29.022066 28496 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:16:29.022336 28496 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:16:29.028136 28496 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffcf3a6c4cef466da32c55eb60acf4fb: Generated new TSK 0
I20260924 21:16:29.028465 28496 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:16:29.038465 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 28403
2026-09-24T21:16:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1309 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:16:29 dist-test-slave-q276 krb5kdc[28502](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:16:29 dist-test-slave-q276 krb5kdc[28502](info): set up 2 sockets
Sep 24 21:16:29 dist-test-slave-q276 krb5kdc[28502](info): commencing operation
krb5kdc: starting...
W20260924 21:16:30.162791 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:16:30 dist-test-slave-q276 krb5kdc[28502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:16:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:16:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.210.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
I20260924 21:16:30.281119 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:39379
--webserver_interface=127.6.210.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:45797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:39379
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:30.370925 28515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:30.371150 28515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:30.371176 28515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:30.372438 28515 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:30.372479 28515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:30.372493 28515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:30.372547 28515 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:30.372572 28515 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:30.374054 28515 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:45797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:39379
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:39379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:30.374333 28515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:30.374642 28515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:30.379658 28521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:30.379766 28523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:30.379806 28520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:30.379846 28515 server_base.cc:1034] running on GCE node
I20260924 21:16:30.380067 28515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:30.380349 28515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:30.381515 28515 hybrid_clock.cc:648] HybridClock initialized: now 1790284590381503 us; error 49 us; skew 500 ppm
Sep 24 21:16:30 dist-test-slave-q276 krb5kdc[28502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284590, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:30.390086 28515 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:16:30.390689 28515 webserver.cc:460] Webserver started at http://127.6.210.190:39201/ using document root <none> and password file <none>
I20260924 21:16:30.390930 28515 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:30.390998 28515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:30.391168 28515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:30.392287 28515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "d88abdcaccaf4dcd8d1912b98c565240"
format_stamp: "Formatted at 2026-09-24 21:16:30 on dist-test-slave-q276"
server_key: "cbf054457e0defbf73911607180235e2"
server_key_iv: "b8eeb886c976af8f7bc81c6ae70c8757"
server_key_version: "encryptionkey@0"
I20260924 21:16:30.392671 28515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "d88abdcaccaf4dcd8d1912b98c565240"
format_stamp: "Formatted at 2026-09-24 21:16:30 on dist-test-slave-q276"
server_key: "cbf054457e0defbf73911607180235e2"
server_key_iv: "b8eeb886c976af8f7bc81c6ae70c8757"
server_key_version: "encryptionkey@0"
I20260924 21:16:30.394023 28515 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:16:30.395016 28530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:30.395236 28515 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 21:16:30.395328 28515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "d88abdcaccaf4dcd8d1912b98c565240"
format_stamp: "Formatted at 2026-09-24 21:16:30 on dist-test-slave-q276"
server_key: "cbf054457e0defbf73911607180235e2"
server_key_iv: "b8eeb886c976af8f7bc81c6ae70c8757"
server_key_version: "encryptionkey@0"
I20260924 21:16:30.395386 28515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:30.419920 28515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:30.422221 28515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:30.422425 28515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:30.426298 28515 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:39379
I20260924 21:16:30.426379 28582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:39379 every 8 connection(s)
I20260924 21:16:30.427006 28515 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:16:30.428026 28583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:30.430439 28583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d88abdcaccaf4dcd8d1912b98c565240: Bootstrap starting.
I20260924 21:16:30.431037 28583 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d88abdcaccaf4dcd8d1912b98c565240: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:30.431320 28583 log.cc:824] T 00000000000000000000000000000000 P d88abdcaccaf4dcd8d1912b98c565240: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:30.432062 28583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d88abdcaccaf4dcd8d1912b98c565240: No bootstrap required, opened a new log
I20260924 21:16:30.433759 28583 raft_consensus.cc:348] T 00000000000000000000000000000000 P d88abdcaccaf4dcd8d1912b98c565240 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d88abdcaccaf4dcd8d1912b98c565240" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39379 } }
I20260924 21:16:30.433902 28583 raft_consensus.cc:374] T 00000000000000000000000000000000 P d88abdcaccaf4dcd8d1912b98c565240 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:30.433924 28583 raft_consensus.cc:729] T 00000000000000000000000000000000 P d88abdcaccaf4dcd8d1912b98c565240 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d88abdcaccaf4dcd8d1912b98c565240, State: Initialized, Role: FOLLOWER
I20260924 21:16:30.434078 28583 consensus_queue.cc:260] T 00000000000000000000000000000000 P d88abdcaccaf4dcd8d1912b98c565240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d88abdcaccaf4dcd8d1912b98c565240" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39379 } }
I20260924 21:16:30.434146 28583 raft_consensus.cc:388] T 00000000000000000000000000000000 P d88abdcaccaf4dcd8d1912b98c565240 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:16:30.434167 28583 raft_consensus.cc:482] T 00000000000000000000000000000000 P d88abdcaccaf4dcd8d1912b98c565240 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:16:30.434195 28583 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d88abdcaccaf4dcd8d1912b98c565240 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:30.435091 28583 raft_consensus.cc:504] T 00000000000000000000000000000000 P d88abdcaccaf4dcd8d1912b98c565240 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d88abdcaccaf4dcd8d1912b98c565240" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39379 } }
I20260924 21:16:30.435179 28583 leader_election.cc:302] T 00000000000000000000000000000000 P d88abdcaccaf4dcd8d1912b98c565240 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d88abdcaccaf4dcd8d1912b98c565240; no voters:
I20260924 21:16:30.435370 28583 leader_election.cc:288] T 00000000000000000000000000000000 P d88abdcaccaf4dcd8d1912b98c565240 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:16:30.435421 28586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d88abdcaccaf4dcd8d1912b98c565240 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:30.435631 28583 sys_catalog.cc:564] T 00000000000000000000000000000000 P d88abdcaccaf4dcd8d1912b98c565240 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:30.435678 28586 raft_consensus.cc:686] T 00000000000000000000000000000000 P d88abdcaccaf4dcd8d1912b98c565240 [term 1 LEADER]: Becoming Leader. State: Replica: d88abdcaccaf4dcd8d1912b98c565240, State: Running, Role: LEADER
I20260924 21:16:30.435806 28586 consensus_queue.cc:237] T 00000000000000000000000000000000 P d88abdcaccaf4dcd8d1912b98c565240 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d88abdcaccaf4dcd8d1912b98c565240" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39379 } }
I20260924 21:16:30.436262 28587 sys_catalog.cc:455] T 00000000000000000000000000000000 P d88abdcaccaf4dcd8d1912b98c565240 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d88abdcaccaf4dcd8d1912b98c565240" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d88abdcaccaf4dcd8d1912b98c565240" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39379 } } }
I20260924 21:16:30.436353 28587 sys_catalog.cc:458] T 00000000000000000000000000000000 P d88abdcaccaf4dcd8d1912b98c565240 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:30.436415 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 28515
I20260924 21:16:30.436529 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:16:30.436515 28588 sys_catalog.cc:455] T 00000000000000000000000000000000 P d88abdcaccaf4dcd8d1912b98c565240 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d88abdcaccaf4dcd8d1912b98c565240. Latest consensus state: current_term: 1 leader_uuid: "d88abdcaccaf4dcd8d1912b98c565240" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d88abdcaccaf4dcd8d1912b98c565240" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 39379 } } }
I20260924 21:16:30.436563 28588 sys_catalog.cc:458] T 00000000000000000000000000000000 P d88abdcaccaf4dcd8d1912b98c565240 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:30.436800 6986 external_mini_cluster.cc:1428] Setting key e1da7e6f5427c59559bb3c2d32281fc8
I20260924 21:16:30.437085 28595 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:16:30.437760 28595 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:16:30.440126 28595 catalog_manager.cc:1347] Generated new cluster ID: f2fc97baf6b14db58defb010305120c9
I20260924 21:16:30.440213 28595 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 21:16:30 dist-test-slave-q276 krb5kdc[28502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:16:30.449215 28604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:30.437305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36335 (local address 127.6.210.190:39379)
0924 21:16:30.437525 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:30.437536 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:30.437576 (+ 40us) server_negotiation.cc:407] Connection header received
0924 21:16:30.437703 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:30.437710 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:30.437763 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:30.437905 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:30.438712 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:30.439997 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:30.441109 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:30.441804 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:30.441930 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:30.444934 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:30.444939 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:30.444940 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:30.444955 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:30.447197 (+ 2242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:30.447703 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:30.447706 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:30.447707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:30.447860 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:30.448152 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:30.448153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:30.448153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:30.448522 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:30.448697 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:30.448846 (+ 149us) server_negotiation.cc:299] Negotiation successful
0924 21:16:30.448962 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":81,"threads_started":1}
I20260924 21:16:30.450714 6986 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:16:30.453030 28595 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:16:30.453600 28595 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:16:30.460300 28604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:30.452389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53659 (local address 127.6.210.190:39379)
0924 21:16:30.452424 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:30.452426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:30.452450 (+ 24us) server_negotiation.cc:407] Connection header received
0924 21:16:30.452521 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:30.452521 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:30.452532 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:30.452580 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:30.453291 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:30.454079 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:30.455228 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:30.455545 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:30.455585 (+ 40us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:30.457238 (+ 1653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:30.457240 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:30.457241 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:30.457247 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:30.458933 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:30.459392 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:30.459393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:30.459394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:30.459447 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:30.459677 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:30.459678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:30.459678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:30.459816 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:30.459894 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:30.460130 (+ 236us) server_negotiation.cc:299] Negotiation successful
0924 21:16:30.460196 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 21:16:30.463147 28595 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d88abdcaccaf4dcd8d1912b98c565240: Generated new TSK 0
I20260924 21:16:30.463347 28595 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:16:30.466391 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 28515
W20260924 21:16:30.470834 28500 connection.cc:535] client connection to 127.6.210.190:39379 recv error: Network error: failed to read from TLS socket (remote: 127.6.210.190:39379): Connection reset by peer (error 104)
2026-09-24T21:16:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1428 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:16:30 dist-test-slave-q276 krb5kdc[28614](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:16:30 dist-test-slave-q276 krb5kdc[28614](info): set up 2 sockets
Sep 24 21:16:30 dist-test-slave-q276 krb5kdc[28614](info): commencing operation
krb5kdc: starting...
W20260924 21:16:31.538370 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:16:31 dist-test-slave-q276 krb5kdc[28614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:16:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:16:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.210.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
I20260924 21:16:31.670173 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:33667
--webserver_interface=127.6.210.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:40413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:33667
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:31.751519 28627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:31.751786 28627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:31.751825 28627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:31.752985 28627 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:31.753054 28627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:31.753082 28627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:31.753114 28627 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:31.753146 28627 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:31.754752 28627 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:40413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:33667
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:33667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:31.755029 28627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:31.755257 28627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:31.760179 28633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:31.760176 28632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:31.760176 28635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:31.760634 28627 server_base.cc:1034] running on GCE node
I20260924 21:16:31.760878 28627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:31.761147 28627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:31.762357 28627 hybrid_clock.cc:648] HybridClock initialized: now 1790284591762330 us; error 33 us; skew 500 ppm
Sep 24 21:16:31 dist-test-slave-q276 krb5kdc[28614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284591, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:31.765154 28627 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:16:31.765744 28627 webserver.cc:460] Webserver started at http://127.6.210.190:32805/ using document root <none> and password file <none>
I20260924 21:16:31.765944 28627 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:31.765990 28627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:31.766090 28627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:31.767324 28627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "5b2eb96eb0544f61be413e5cc7111e68"
format_stamp: "Formatted at 2026-09-24 21:16:31 on dist-test-slave-q276"
server_key: "66cde6158f2db9251837b2143e4f013c"
server_key_iv: "327b68fffd31ef4e2355a91857a7d3bc"
server_key_version: "encryptionkey@0"
I20260924 21:16:31.767679 28627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "5b2eb96eb0544f61be413e5cc7111e68"
format_stamp: "Formatted at 2026-09-24 21:16:31 on dist-test-slave-q276"
server_key: "66cde6158f2db9251837b2143e4f013c"
server_key_iv: "327b68fffd31ef4e2355a91857a7d3bc"
server_key_version: "encryptionkey@0"
I20260924 21:16:31.769039 28627 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:16:31.769802 28642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:31.769970 28627 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:31.770061 28627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "5b2eb96eb0544f61be413e5cc7111e68"
format_stamp: "Formatted at 2026-09-24 21:16:31 on dist-test-slave-q276"
server_key: "66cde6158f2db9251837b2143e4f013c"
server_key_iv: "327b68fffd31ef4e2355a91857a7d3bc"
server_key_version: "encryptionkey@0"
I20260924 21:16:31.770138 28627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:31.787406 28627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:31.789662 28627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:31.789777 28627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:31.793506 28627 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:33667
I20260924 21:16:31.793579 28694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:33667 every 8 connection(s)
I20260924 21:16:31.793916 28627 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:16:31.794811 28695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:31.795009 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 28627
I20260924 21:16:31.795145 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:16:31.795333 6986 external_mini_cluster.cc:1428] Setting key 4ce7cc3fa507930f321d983e14652b16
I20260924 21:16:31.798393 28695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b2eb96eb0544f61be413e5cc7111e68: Bootstrap starting.
I20260924 21:16:31.799000 28695 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b2eb96eb0544f61be413e5cc7111e68: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:31.799333 28695 log.cc:824] T 00000000000000000000000000000000 P 5b2eb96eb0544f61be413e5cc7111e68: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:31.800225 28695 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b2eb96eb0544f61be413e5cc7111e68: No bootstrap required, opened a new log
I20260924 21:16:31.802191 28698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:31.795836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40027 (local address 127.6.210.190:33667)
0924 21:16:31.796069 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:31.796078 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:31.796108 (+ 30us) server_negotiation.cc:407] Connection header received
0924 21:16:31.796570 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:31.796580 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:31.796644 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:31.796795 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:31.796991 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:31.798372 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:31.799948 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:31.800571 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:31.800695 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:31.801466 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:31.801471 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:31.801471 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:31.801666 (+ 195us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:31.801702 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:31.801717 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:31.801816 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:31.801912 (+ 96us) server_negotiation.cc:299] Negotiation successful
0924 21:16:31.801941 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":58,"threads_started":1}
I20260924 21:16:31.802623 28695 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b2eb96eb0544f61be413e5cc7111e68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b2eb96eb0544f61be413e5cc7111e68" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 33667 } }
I20260924 21:16:31.802785 28695 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b2eb96eb0544f61be413e5cc7111e68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:31.802820 28695 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b2eb96eb0544f61be413e5cc7111e68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b2eb96eb0544f61be413e5cc7111e68, State: Initialized, Role: FOLLOWER
I20260924 21:16:31.803004 28695 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b2eb96eb0544f61be413e5cc7111e68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b2eb96eb0544f61be413e5cc7111e68" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 33667 } }
I20260924 21:16:31.803103 28695 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b2eb96eb0544f61be413e5cc7111e68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:16:31.803172 28695 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b2eb96eb0544f61be413e5cc7111e68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:16:31.803226 28695 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b2eb96eb0544f61be413e5cc7111e68 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:31.803361 6986 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:16:31.803755 28695 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b2eb96eb0544f61be413e5cc7111e68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b2eb96eb0544f61be413e5cc7111e68" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 33667 } }
I20260924 21:16:31.803838 28695 leader_election.cc:302] T 00000000000000000000000000000000 P 5b2eb96eb0544f61be413e5cc7111e68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b2eb96eb0544f61be413e5cc7111e68; no voters:
I20260924 21:16:31.803960 28695 leader_election.cc:288] T 00000000000000000000000000000000 P 5b2eb96eb0544f61be413e5cc7111e68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:16:31.804020 28700 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b2eb96eb0544f61be413e5cc7111e68 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:31.804179 28700 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b2eb96eb0544f61be413e5cc7111e68 [term 1 LEADER]: Becoming Leader. State: Replica: 5b2eb96eb0544f61be413e5cc7111e68, State: Running, Role: LEADER
I20260924 21:16:31.804211 28695 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b2eb96eb0544f61be413e5cc7111e68 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:31.804337 28700 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b2eb96eb0544f61be413e5cc7111e68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b2eb96eb0544f61be413e5cc7111e68" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 33667 } }
I20260924 21:16:31.804890 28702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b2eb96eb0544f61be413e5cc7111e68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b2eb96eb0544f61be413e5cc7111e68. Latest consensus state: current_term: 1 leader_uuid: "5b2eb96eb0544f61be413e5cc7111e68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b2eb96eb0544f61be413e5cc7111e68" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 33667 } } }
I20260924 21:16:31.804967 28702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b2eb96eb0544f61be413e5cc7111e68 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:31.805169 28701 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b2eb96eb0544f61be413e5cc7111e68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b2eb96eb0544f61be413e5cc7111e68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b2eb96eb0544f61be413e5cc7111e68" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 33667 } } }
I20260924 21:16:31.805457 28701 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b2eb96eb0544f61be413e5cc7111e68 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:31.805696 28713 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:16:31.806186 28713 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:16:31.807703 28713 catalog_manager.cc:1347] Generated new cluster ID: e58404fc475c452b8f91e2314e333bda
I20260924 21:16:31.807754 28713 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:16:31.808651 28698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:31.804922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35743 (local address 127.6.210.190:33667)
0924 21:16:31.804961 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:31.804963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:31.805071 (+ 108us) server_negotiation.cc:407] Connection header received
0924 21:16:31.805172 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:31.805174 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:31.805185 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:31.805216 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:31.805447 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:31.806203 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:31.807302 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:31.807602 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:31.807678 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:31.808376 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:31.808377 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:31.808378 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:31.808382 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:31.808399 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:31.808403 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:31.808460 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:31.808553 (+ 93us) server_negotiation.cc:299] Negotiation successful
0924 21:16:31.808575 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 21:16:31.815369 28713 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:16:31.815900 28713 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:16:31.819511 28713 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b2eb96eb0544f61be413e5cc7111e68: Generated new TSK 0
I20260924 21:16:31.819727 28713 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:16:31.825850 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 28627
W20260924 21:16:31.830101 28612 connection.cc:535] client connection to 127.6.210.190:33667 recv error: Network error: failed to read from TLS socket (remote: 127.6.210.190:33667): Connection reset by peer (error 104)
2026-09-24T21:16:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1359 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:16:31 dist-test-slave-q276 krb5kdc[28726](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:16:31 dist-test-slave-q276 krb5kdc[28726](info): set up 2 sockets
Sep 24 21:16:31 dist-test-slave-q276 krb5kdc[28726](info): commencing operation
krb5kdc: starting...
W20260924 21:16:32.859004 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:16:32 dist-test-slave-q276 krb5kdc[28726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:16:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:16:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.210.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
I20260924 21:16:32.979291 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:45955
--webserver_interface=127.6.210.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:38971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:45955
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:33.060482 28739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:33.060745 28739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:33.060796 28739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:33.062028 28739 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:33.062106 28739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:33.062131 28739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:33.062163 28739 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:33.062191 28739 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:33.063737 28739 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:38971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:45955
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:45955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:33.064041 28739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:33.064287 28739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 21:16:33.069664 28739 server_base.cc:1034] running on GCE node
W20260924 21:16:33.069621 28747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:33.069587 28745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:33.069607 28744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:33.070076 28739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:33.070488 28739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:33.071663 28739 hybrid_clock.cc:648] HybridClock initialized: now 1790284593071665 us; error 45 us; skew 500 ppm
Sep 24 21:16:33 dist-test-slave-q276 krb5kdc[28726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284593, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:33.074487 28739 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:16:33.074966 28739 webserver.cc:460] Webserver started at http://127.6.210.190:45689/ using document root <none> and password file <none>
I20260924 21:16:33.075244 28739 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:33.075317 28739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:33.075467 28739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:33.076647 28739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "6c06bb0bcfc14493925a2c697b4baceb"
format_stamp: "Formatted at 2026-09-24 21:16:33 on dist-test-slave-q276"
server_key: "649e4e2e75db3cb8c95c4fecc66be379"
server_key_iv: "a3e17785adc0ab873eccb1a996433f04"
server_key_version: "encryptionkey@0"
I20260924 21:16:33.077028 28739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "6c06bb0bcfc14493925a2c697b4baceb"
format_stamp: "Formatted at 2026-09-24 21:16:33 on dist-test-slave-q276"
server_key: "649e4e2e75db3cb8c95c4fecc66be379"
server_key_iv: "a3e17785adc0ab873eccb1a996433f04"
server_key_version: "encryptionkey@0"
I20260924 21:16:33.078540 28739 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:16:33.079411 28754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:33.079634 28739 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:33.079748 28739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "6c06bb0bcfc14493925a2c697b4baceb"
format_stamp: "Formatted at 2026-09-24 21:16:33 on dist-test-slave-q276"
server_key: "649e4e2e75db3cb8c95c4fecc66be379"
server_key_iv: "a3e17785adc0ab873eccb1a996433f04"
server_key_version: "encryptionkey@0"
I20260924 21:16:33.079846 28739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:33.083267 28739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:33.085196 28739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:33.085353 28739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:33.089494 28739 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:45955
I20260924 21:16:33.089527 28806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:45955 every 8 connection(s)
I20260924 21:16:33.089864 28739 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:16:33.090951 28807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:33.093248 28807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c06bb0bcfc14493925a2c697b4baceb: Bootstrap starting.
I20260924 21:16:33.093746 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 28739
I20260924 21:16:33.093852 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:16:33.093979 28807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c06bb0bcfc14493925a2c697b4baceb: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:33.094049 6986 external_mini_cluster.cc:1428] Setting key 4eb464045ff11692e37665c6ec41c953
I20260924 21:16:33.094353 28807 log.cc:824] T 00000000000000000000000000000000 P 6c06bb0bcfc14493925a2c697b4baceb: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:33.095115 28807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c06bb0bcfc14493925a2c697b4baceb: No bootstrap required, opened a new log
I20260924 21:16:33.096709 28811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:33.094574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39263 (local address 127.6.210.190:45955)
0924 21:16:33.094738 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:33.094749 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:33.094811 (+ 62us) server_negotiation.cc:407] Connection header received
0924 21:16:33.095427 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:33.095434 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:33.095463 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:33.095612 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:33.096297 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:33.096301 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:33.096301 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:33.096306 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:33.096352 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:33.096371 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:33.096463 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:33.096501 (+ 38us) server_negotiation.cc:299] Negotiation successful
0924 21:16:33.096541 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":69,"threads_started":1}
I20260924 21:16:33.097479 28807 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c06bb0bcfc14493925a2c697b4baceb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c06bb0bcfc14493925a2c697b4baceb" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 45955 } }
I20260924 21:16:33.097656 28807 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c06bb0bcfc14493925a2c697b4baceb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:33.097702 28807 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c06bb0bcfc14493925a2c697b4baceb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c06bb0bcfc14493925a2c697b4baceb, State: Initialized, Role: FOLLOWER
I20260924 21:16:33.097842 6986 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:16:33.097832 28807 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c06bb0bcfc14493925a2c697b4baceb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c06bb0bcfc14493925a2c697b4baceb" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 45955 } }
I20260924 21:16:33.097936 28807 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c06bb0bcfc14493925a2c697b4baceb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:16:33.097980 28807 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c06bb0bcfc14493925a2c697b4baceb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:16:33.098029 28807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c06bb0bcfc14493925a2c697b4baceb [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:33.098649 28807 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c06bb0bcfc14493925a2c697b4baceb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c06bb0bcfc14493925a2c697b4baceb" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 45955 } }
I20260924 21:16:33.098778 28807 leader_election.cc:302] T 00000000000000000000000000000000 P 6c06bb0bcfc14493925a2c697b4baceb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c06bb0bcfc14493925a2c697b4baceb; no voters:
I20260924 21:16:33.098942 28807 leader_election.cc:288] T 00000000000000000000000000000000 P 6c06bb0bcfc14493925a2c697b4baceb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:16:33.098973 28815 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c06bb0bcfc14493925a2c697b4baceb [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:33.099170 28815 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c06bb0bcfc14493925a2c697b4baceb [term 1 LEADER]: Becoming Leader. State: Replica: 6c06bb0bcfc14493925a2c697b4baceb, State: Running, Role: LEADER
I20260924 21:16:33.099274 28815 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c06bb0bcfc14493925a2c697b4baceb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c06bb0bcfc14493925a2c697b4baceb" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 45955 } }
I20260924 21:16:33.099303 28807 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c06bb0bcfc14493925a2c697b4baceb [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:33.099759 28817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c06bb0bcfc14493925a2c697b4baceb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c06bb0bcfc14493925a2c697b4baceb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c06bb0bcfc14493925a2c697b4baceb" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 45955 } } }
I20260924 21:16:33.099777 28818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c06bb0bcfc14493925a2c697b4baceb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c06bb0bcfc14493925a2c697b4baceb. Latest consensus state: current_term: 1 leader_uuid: "6c06bb0bcfc14493925a2c697b4baceb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c06bb0bcfc14493925a2c697b4baceb" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 45955 } } }
I20260924 21:16:33.099925 28818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c06bb0bcfc14493925a2c697b4baceb [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:33.099915 28817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c06bb0bcfc14493925a2c697b4baceb [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:33.100342 28825 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:16:33.100852 28825 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:16:33.101305 28811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:33.099331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37931 (local address 127.6.210.190:45955)
0924 21:16:33.099380 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:33.099382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:33.099887 (+ 505us) server_negotiation.cc:407] Connection header received
0924 21:16:33.100242 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:33.100243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:33.100248 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:33.100298 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:33.100989 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:33.100990 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:33.100991 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:33.100995 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:33.101009 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:33.101012 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:33.101055 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:33.101241 (+ 186us) server_negotiation.cc:299] Negotiation successful
0924 21:16:33.101252 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260924 21:16:33.102617 28825 catalog_manager.cc:1347] Generated new cluster ID: e3e17668b6a74dc3a3c8e4ec38f2e89b
I20260924 21:16:33.102661 28825 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:16:33.120290 28825 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:16:33.120497 28825 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:16:33.127192 28825 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c06bb0bcfc14493925a2c697b4baceb: Generated new TSK 0
I20260924 21:16:33.127417 28825 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:16:33.131675 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 28739
2026-09-24T21:16:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1305 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:16:33 dist-test-slave-q276 krb5kdc[28838](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:16:33 dist-test-slave-q276 krb5kdc[28838](info): set up 2 sockets
Sep 24 21:16:33 dist-test-slave-q276 krb5kdc[28838](info): commencing operation
krb5kdc: starting...
W20260924 21:16:34.242352 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:16:34 dist-test-slave-q276 krb5kdc[28838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:16:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:16:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.210.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
I20260924 21:16:34.363212 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:37937
--webserver_interface=127.6.210.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:45407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:37937
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:34.444700 28851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:34.444912 28851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:34.444936 28851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:34.446100 28851 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:34.446146 28851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:34.446161 28851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:34.446173 28851 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:34.446229 28851 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:34.447822 28851 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:45407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:37937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:37937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:34.448077 28851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:34.448369 28851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:34.453622 28859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:34.453856 28851 server_base.cc:1034] running on GCE node
W20260924 21:16:34.453620 28856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:34.453886 28857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:34.454178 28851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:34.454541 28851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:34.455693 28851 hybrid_clock.cc:648] HybridClock initialized: now 1790284594455682 us; error 31 us; skew 500 ppm
Sep 24 21:16:34 dist-test-slave-q276 krb5kdc[28838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284594, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:34.458250 28851 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:16:34.458806 28851 webserver.cc:460] Webserver started at http://127.6.210.190:45295/ using document root <none> and password file <none>
I20260924 21:16:34.459033 28851 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:34.459100 28851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:34.459255 28851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:34.460351 28851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "1e13170ff1b74d0b9dd331dae4099af1"
format_stamp: "Formatted at 2026-09-24 21:16:34 on dist-test-slave-q276"
server_key: "f5848937b1c5bfd4e5ee88c6662edb63"
server_key_iv: "e9313e74a77c9e2f6f4846ecd01f892c"
server_key_version: "encryptionkey@0"
I20260924 21:16:34.460721 28851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "1e13170ff1b74d0b9dd331dae4099af1"
format_stamp: "Formatted at 2026-09-24 21:16:34 on dist-test-slave-q276"
server_key: "f5848937b1c5bfd4e5ee88c6662edb63"
server_key_iv: "e9313e74a77c9e2f6f4846ecd01f892c"
server_key_version: "encryptionkey@0"
I20260924 21:16:34.462054 28851 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260924 21:16:34.462839 28866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:34.463006 28851 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:34.463101 28851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "1e13170ff1b74d0b9dd331dae4099af1"
format_stamp: "Formatted at 2026-09-24 21:16:34 on dist-test-slave-q276"
server_key: "f5848937b1c5bfd4e5ee88c6662edb63"
server_key_iv: "e9313e74a77c9e2f6f4846ecd01f892c"
server_key_version: "encryptionkey@0"
I20260924 21:16:34.463184 28851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:34.474854 28851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:34.477154 28851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:34.477311 28851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:34.481392 28851 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:37937
I20260924 21:16:34.481446 28918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:37937 every 8 connection(s)
I20260924 21:16:34.481814 28851 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:16:34.482743 28919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:34.484920 28919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e13170ff1b74d0b9dd331dae4099af1: Bootstrap starting.
I20260924 21:16:34.485496 28919 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e13170ff1b74d0b9dd331dae4099af1: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:34.485776 28919 log.cc:824] T 00000000000000000000000000000000 P 1e13170ff1b74d0b9dd331dae4099af1: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:34.486517 28919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e13170ff1b74d0b9dd331dae4099af1: No bootstrap required, opened a new log
I20260924 21:16:34.488041 28919 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e13170ff1b74d0b9dd331dae4099af1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e13170ff1b74d0b9dd331dae4099af1" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37937 } }
I20260924 21:16:34.488215 28919 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e13170ff1b74d0b9dd331dae4099af1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:34.488260 28919 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e13170ff1b74d0b9dd331dae4099af1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e13170ff1b74d0b9dd331dae4099af1, State: Initialized, Role: FOLLOWER
I20260924 21:16:34.488353 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 28851
I20260924 21:16:34.488457 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:16:34.488418 28919 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e13170ff1b74d0b9dd331dae4099af1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e13170ff1b74d0b9dd331dae4099af1" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37937 } }
I20260924 21:16:34.488502 28919 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e13170ff1b74d0b9dd331dae4099af1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:16:34.488544 28919 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e13170ff1b74d0b9dd331dae4099af1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:16:34.488600 28919 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e13170ff1b74d0b9dd331dae4099af1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:34.488637 6986 external_mini_cluster.cc:1428] Setting key dfaea31d9bef95fecfc4a2ec4c04f149
I20260924 21:16:34.489571 28919 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e13170ff1b74d0b9dd331dae4099af1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e13170ff1b74d0b9dd331dae4099af1" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37937 } }
I20260924 21:16:34.489658 28919 leader_election.cc:302] T 00000000000000000000000000000000 P 1e13170ff1b74d0b9dd331dae4099af1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e13170ff1b74d0b9dd331dae4099af1; no voters:
I20260924 21:16:34.489822 28919 leader_election.cc:288] T 00000000000000000000000000000000 P 1e13170ff1b74d0b9dd331dae4099af1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:16:34.489854 28924 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e13170ff1b74d0b9dd331dae4099af1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:34.490020 28924 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e13170ff1b74d0b9dd331dae4099af1 [term 1 LEADER]: Becoming Leader. State: Replica: 1e13170ff1b74d0b9dd331dae4099af1, State: Running, Role: LEADER
I20260924 21:16:34.490078 28919 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e13170ff1b74d0b9dd331dae4099af1 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:34.490157 28924 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e13170ff1b74d0b9dd331dae4099af1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e13170ff1b74d0b9dd331dae4099af1" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37937 } }
I20260924 21:16:34.490571 28924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e13170ff1b74d0b9dd331dae4099af1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e13170ff1b74d0b9dd331dae4099af1. Latest consensus state: current_term: 1 leader_uuid: "1e13170ff1b74d0b9dd331dae4099af1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e13170ff1b74d0b9dd331dae4099af1" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37937 } } }
I20260924 21:16:34.490721 28924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e13170ff1b74d0b9dd331dae4099af1 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:34.490985 28933 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:16:34.491523 28925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e13170ff1b74d0b9dd331dae4099af1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e13170ff1b74d0b9dd331dae4099af1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e13170ff1b74d0b9dd331dae4099af1" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 37937 } } }
I20260924 21:16:34.491658 28925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e13170ff1b74d0b9dd331dae4099af1 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:34.491827 28933 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:16:34.493611 28933 catalog_manager.cc:1347] Generated new cluster ID: 1888c92027ac4c139a42972f32bdcf3f
I20260924 21:16:34.493669 28933 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 21:16:34 dist-test-slave-q276 krb5kdc[28838](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:16:34.499881 28933 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:16:34.500628 28933 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:16:34.502015 28923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:34.489155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52977 (local address 127.6.210.190:37937)
0924 21:16:34.489372 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:34.489384 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:34.489420 (+ 36us) server_negotiation.cc:407] Connection header received
0924 21:16:34.489534 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:34.489541 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:34.489585 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:34.489758 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:34.490561 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:34.492201 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:34.493281 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:34.493882 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:34.493993 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:34.497146 (+ 3153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:34.497150 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:34.497150 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:34.497163 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:34.500279 (+ 3116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:34.500700 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:34.500703 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:34.500704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:34.500819 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:34.501048 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:34.501049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:34.501049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:34.501339 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:34.501486 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:34.501602 (+ 116us) server_negotiation.cc:299] Negotiation successful
0924 21:16:34.501761 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":110,"threads_started":1}
I20260924 21:16:34.503011 6986 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:16:34.507658 28933 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e13170ff1b74d0b9dd331dae4099af1: Generated new TSK 0
I20260924 21:16:34.508263 28933 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:16:34.511659 28923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:34.504414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55771 (local address 127.6.210.190:37937)
0924 21:16:34.504498 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:34.504500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:34.504549 (+ 49us) server_negotiation.cc:407] Connection header received
0924 21:16:34.504611 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:34.504612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:34.504621 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:34.504676 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:34.505270 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:34.506065 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:34.507154 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:34.507468 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:34.507541 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:34.508987 (+ 1446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:34.508988 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:34.508988 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:34.508995 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:34.510528 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:34.510891 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:34.510892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:34.510893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:34.510951 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:34.511180 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:34.511181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:34.511181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:34.511307 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:34.511402 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:34.511494 (+ 92us) server_negotiation.cc:299] Negotiation successful
0924 21:16:34.511571 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260924 21:16:34.513597 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 28851
2026-09-24T21:16:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1381 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:16:34 dist-test-slave-q276 krb5kdc[28950](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:16:34 dist-test-slave-q276 krb5kdc[28950](info): set up 2 sockets
Sep 24 21:16:34 dist-test-slave-q276 krb5kdc[28950](info): commencing operation
krb5kdc: starting...
W20260924 21:16:35.550531 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:16:35 dist-test-slave-q276 krb5kdc[28950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:16:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:16:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.210.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
I20260924 21:16:35.669309 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:46059
--webserver_interface=127.6.210.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:34017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:46059
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:35.748241 28963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:35.748471 28963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:35.748510 28963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:35.749632 28963 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:35.749694 28963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:35.749716 28963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:35.749737 28963 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:35.749765 28963 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:35.751225 28963 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:34017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:46059
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:46059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:35.751487 28963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:35.751757 28963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:35.756417 28969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:35.756425 28968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:35.756435 28971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:35.756606 28963 server_base.cc:1034] running on GCE node
I20260924 21:16:35.756875 28963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:35.757161 28963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:35.758332 28963 hybrid_clock.cc:648] HybridClock initialized: now 1790284595758313 us; error 44 us; skew 500 ppm
Sep 24 21:16:35 dist-test-slave-q276 krb5kdc[28950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284595, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:35.764411 28963 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:16:35.764874 28963 webserver.cc:460] Webserver started at http://127.6.210.190:35633/ using document root <none> and password file <none>
I20260924 21:16:35.765107 28963 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:35.765153 28963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:35.765272 28963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:35.766388 28963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "b292c59430dd4951aed1025d3b1e8f4c"
format_stamp: "Formatted at 2026-09-24 21:16:35 on dist-test-slave-q276"
server_key: "164bc477abcf5abeb4eb7efb353669f3"
server_key_iv: "fdaa29c7ffa097b922aa582978b10c06"
server_key_version: "encryptionkey@0"
I20260924 21:16:35.766736 28963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "b292c59430dd4951aed1025d3b1e8f4c"
format_stamp: "Formatted at 2026-09-24 21:16:35 on dist-test-slave-q276"
server_key: "164bc477abcf5abeb4eb7efb353669f3"
server_key_iv: "fdaa29c7ffa097b922aa582978b10c06"
server_key_version: "encryptionkey@0"
I20260924 21:16:35.768033 28963 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:16:35.768811 28978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:35.769027 28963 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:16:35.769120 28963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "b292c59430dd4951aed1025d3b1e8f4c"
format_stamp: "Formatted at 2026-09-24 21:16:35 on dist-test-slave-q276"
server_key: "164bc477abcf5abeb4eb7efb353669f3"
server_key_iv: "fdaa29c7ffa097b922aa582978b10c06"
server_key_version: "encryptionkey@0"
I20260924 21:16:35.769178 28963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:35.780380 28963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:35.782166 28963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:35.782370 28963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:35.785902 28963 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:46059
I20260924 21:16:35.785951 29030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:46059 every 8 connection(s)
I20260924 21:16:35.786334 28963 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:16:35.787170 29031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:35.789330 29031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b292c59430dd4951aed1025d3b1e8f4c: Bootstrap starting.
I20260924 21:16:35.789889 29031 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b292c59430dd4951aed1025d3b1e8f4c: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:35.790169 29031 log.cc:824] T 00000000000000000000000000000000 P b292c59430dd4951aed1025d3b1e8f4c: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:35.790887 29031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b292c59430dd4951aed1025d3b1e8f4c: No bootstrap required, opened a new log
I20260924 21:16:35.792388 29031 raft_consensus.cc:348] T 00000000000000000000000000000000 P b292c59430dd4951aed1025d3b1e8f4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b292c59430dd4951aed1025d3b1e8f4c" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 46059 } }
I20260924 21:16:35.792538 29031 raft_consensus.cc:374] T 00000000000000000000000000000000 P b292c59430dd4951aed1025d3b1e8f4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:35.792584 29031 raft_consensus.cc:729] T 00000000000000000000000000000000 P b292c59430dd4951aed1025d3b1e8f4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b292c59430dd4951aed1025d3b1e8f4c, State: Initialized, Role: FOLLOWER
I20260924 21:16:35.792721 29031 consensus_queue.cc:260] T 00000000000000000000000000000000 P b292c59430dd4951aed1025d3b1e8f4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b292c59430dd4951aed1025d3b1e8f4c" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 46059 } }
I20260924 21:16:35.792795 29031 raft_consensus.cc:388] T 00000000000000000000000000000000 P b292c59430dd4951aed1025d3b1e8f4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:16:35.792845 29031 raft_consensus.cc:482] T 00000000000000000000000000000000 P b292c59430dd4951aed1025d3b1e8f4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:16:35.792894 29031 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b292c59430dd4951aed1025d3b1e8f4c [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:35.793704 29031 raft_consensus.cc:504] T 00000000000000000000000000000000 P b292c59430dd4951aed1025d3b1e8f4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b292c59430dd4951aed1025d3b1e8f4c" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 46059 } }
I20260924 21:16:35.793833 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 28963
I20260924 21:16:35.793819 29031 leader_election.cc:302] T 00000000000000000000000000000000 P b292c59430dd4951aed1025d3b1e8f4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b292c59430dd4951aed1025d3b1e8f4c; no voters:
I20260924 21:16:35.793941 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:16:35.794003 29031 leader_election.cc:288] T 00000000000000000000000000000000 P b292c59430dd4951aed1025d3b1e8f4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:16:35.794037 29034 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b292c59430dd4951aed1025d3b1e8f4c [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:35.794164 6986 external_mini_cluster.cc:1428] Setting key 3c61ee5d81e570949ec154d11f1c43d9
I20260924 21:16:35.794183 29034 raft_consensus.cc:686] T 00000000000000000000000000000000 P b292c59430dd4951aed1025d3b1e8f4c [term 1 LEADER]: Becoming Leader. State: Replica: b292c59430dd4951aed1025d3b1e8f4c, State: Running, Role: LEADER
I20260924 21:16:35.794250 29031 sys_catalog.cc:564] T 00000000000000000000000000000000 P b292c59430dd4951aed1025d3b1e8f4c [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:35.794255 29034 consensus_queue.cc:237] T 00000000000000000000000000000000 P b292c59430dd4951aed1025d3b1e8f4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b292c59430dd4951aed1025d3b1e8f4c" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 46059 } }
I20260924 21:16:35.794616 29035 sys_catalog.cc:455] T 00000000000000000000000000000000 P b292c59430dd4951aed1025d3b1e8f4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b292c59430dd4951aed1025d3b1e8f4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b292c59430dd4951aed1025d3b1e8f4c" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 46059 } } }
I20260924 21:16:35.794708 29035 sys_catalog.cc:458] T 00000000000000000000000000000000 P b292c59430dd4951aed1025d3b1e8f4c [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:35.794883 29036 sys_catalog.cc:455] T 00000000000000000000000000000000 P b292c59430dd4951aed1025d3b1e8f4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b292c59430dd4951aed1025d3b1e8f4c. Latest consensus state: current_term: 1 leader_uuid: "b292c59430dd4951aed1025d3b1e8f4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b292c59430dd4951aed1025d3b1e8f4c" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 46059 } } }
I20260924 21:16:35.794947 29036 sys_catalog.cc:458] T 00000000000000000000000000000000 P b292c59430dd4951aed1025d3b1e8f4c [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:35.795408 29041 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:16:35.795953 29041 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:16:35.797950 29041 catalog_manager.cc:1347] Generated new cluster ID: 83299527b0154dffa9da4e7cde27881a
I20260924 21:16:35.798010 29041 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:16:35.813134 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:35.794818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35619 (local address 127.6.210.190:46059)
0924 21:16:35.795128 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:35.795138 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:35.795300 (+ 162us) server_negotiation.cc:407] Connection header received
0924 21:16:35.795451 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:35.795460 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:35.795527 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:35.795686 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:35.795967 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:35.797408 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:35.810768 (+ 13360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:35.811511 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:35.811642 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:35.812378 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:35.812383 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:35.812383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:35.812617 (+ 234us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:35.812658 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:35.812674 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:35.812743 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:35.812807 (+ 64us) server_negotiation.cc:299] Negotiation successful
0924 21:16:35.812875 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":68,"threads_started":1}
I20260924 21:16:35.815253 6986 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:16:35.816654 29041 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:16:35.817437 29041 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:16:35.820402 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:35.816700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46043 (local address 127.6.210.190:46059)
0924 21:16:35.816739 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:35.816741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:35.816990 (+ 249us) server_negotiation.cc:407] Connection header received
0924 21:16:35.817063 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:35.817064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:35.817075 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:35.817131 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:35.817278 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:35.818038 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:35.819093 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:35.819407 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:35.819443 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:35.820152 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:35.820153 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:35.820154 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:35.820159 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:35.820171 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:35.820175 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:35.820200 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:35.820288 (+ 88us) server_negotiation.cc:299] Negotiation successful
0924 21:16:35.820330 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 21:16:35.823642 29041 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b292c59430dd4951aed1025d3b1e8f4c: Generated new TSK 0
I20260924 21:16:35.823817 29041 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:16:35.825940 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 28963
2026-09-24T21:16:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1312 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:16:35 dist-test-slave-q276 krb5kdc[29062](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:16:35 dist-test-slave-q276 krb5kdc[29062](info): set up 2 sockets
Sep 24 21:16:35 dist-test-slave-q276 krb5kdc[29062](info): commencing operation
krb5kdc: starting...
W20260924 21:16:36.878787 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:16:36 dist-test-slave-q276 krb5kdc[29062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:16:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:16:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.210.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
I20260924 21:16:36.995339 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:34365
--webserver_interface=127.6.210.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:38593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:34365
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:37.081205 29075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:37.081491 29075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:37.081539 29075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:37.082967 29075 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:37.083019 29075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:37.083065 29075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:37.083089 29075 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:37.083123 29075 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:37.084787 29075 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:38593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:34365
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:34365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:37.085088 29075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:37.085340 29075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:37.090576 29083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:37.090610 29080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:37.090566 29081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:37.090871 29075 server_base.cc:1034] running on GCE node
I20260924 21:16:37.091095 29075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:37.091351 29075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:37.092499 29075 hybrid_clock.cc:648] HybridClock initialized: now 1790284597092475 us; error 39 us; skew 500 ppm
Sep 24 21:16:37 dist-test-slave-q276 krb5kdc[29062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284597, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:37.095110 29075 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:16:37.095579 29075 webserver.cc:460] Webserver started at http://127.6.210.190:36979/ using document root <none> and password file <none>
I20260924 21:16:37.095825 29075 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:37.095893 29075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:37.096030 29075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:37.097074 29075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "e56cfa4331af483f82eee6fc130d8839"
format_stamp: "Formatted at 2026-09-24 21:16:37 on dist-test-slave-q276"
server_key: "5d4f065794ed274fddd60ce4e938ffc0"
server_key_iv: "4b6ce0bc437a8ec3315f486fc2a43cda"
server_key_version: "encryptionkey@0"
I20260924 21:16:37.097435 29075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "e56cfa4331af483f82eee6fc130d8839"
format_stamp: "Formatted at 2026-09-24 21:16:37 on dist-test-slave-q276"
server_key: "5d4f065794ed274fddd60ce4e938ffc0"
server_key_iv: "4b6ce0bc437a8ec3315f486fc2a43cda"
server_key_version: "encryptionkey@0"
I20260924 21:16:37.098805 29075 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260924 21:16:37.099529 29090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:37.099701 29075 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:16:37.099802 29075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "e56cfa4331af483f82eee6fc130d8839"
format_stamp: "Formatted at 2026-09-24 21:16:37 on dist-test-slave-q276"
server_key: "5d4f065794ed274fddd60ce4e938ffc0"
server_key_iv: "4b6ce0bc437a8ec3315f486fc2a43cda"
server_key_version: "encryptionkey@0"
I20260924 21:16:37.099900 29075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:37.103267 29075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:37.105500 29075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:37.105648 29075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:37.109530 29075 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:34365
I20260924 21:16:37.109572 29142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:34365 every 8 connection(s)
I20260924 21:16:37.109956 29075 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:16:37.110954 29143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:37.113089 29143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e56cfa4331af483f82eee6fc130d8839: Bootstrap starting.
I20260924 21:16:37.113591 29143 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e56cfa4331af483f82eee6fc130d8839: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:37.113814 29143 log.cc:824] T 00000000000000000000000000000000 P e56cfa4331af483f82eee6fc130d8839: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:37.114485 29143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e56cfa4331af483f82eee6fc130d8839: No bootstrap required, opened a new log
I20260924 21:16:37.116003 29143 raft_consensus.cc:348] T 00000000000000000000000000000000 P e56cfa4331af483f82eee6fc130d8839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e56cfa4331af483f82eee6fc130d8839" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34365 } }
I20260924 21:16:37.116132 29143 raft_consensus.cc:374] T 00000000000000000000000000000000 P e56cfa4331af483f82eee6fc130d8839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:37.116153 29143 raft_consensus.cc:729] T 00000000000000000000000000000000 P e56cfa4331af483f82eee6fc130d8839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e56cfa4331af483f82eee6fc130d8839, State: Initialized, Role: FOLLOWER
I20260924 21:16:37.116248 29143 consensus_queue.cc:260] T 00000000000000000000000000000000 P e56cfa4331af483f82eee6fc130d8839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e56cfa4331af483f82eee6fc130d8839" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34365 } }
I20260924 21:16:37.116292 29143 raft_consensus.cc:388] T 00000000000000000000000000000000 P e56cfa4331af483f82eee6fc130d8839 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:16:37.116312 29143 raft_consensus.cc:482] T 00000000000000000000000000000000 P e56cfa4331af483f82eee6fc130d8839 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:16:37.116336 29143 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e56cfa4331af483f82eee6fc130d8839 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:37.117094 29143 raft_consensus.cc:504] T 00000000000000000000000000000000 P e56cfa4331af483f82eee6fc130d8839 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e56cfa4331af483f82eee6fc130d8839" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34365 } }
I20260924 21:16:37.117177 29143 leader_election.cc:302] T 00000000000000000000000000000000 P e56cfa4331af483f82eee6fc130d8839 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e56cfa4331af483f82eee6fc130d8839; no voters:
I20260924 21:16:37.117286 29143 leader_election.cc:288] T 00000000000000000000000000000000 P e56cfa4331af483f82eee6fc130d8839 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:16:37.117344 29146 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e56cfa4331af483f82eee6fc130d8839 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:37.117506 29143 sys_catalog.cc:564] T 00000000000000000000000000000000 P e56cfa4331af483f82eee6fc130d8839 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:37.117512 29146 raft_consensus.cc:686] T 00000000000000000000000000000000 P e56cfa4331af483f82eee6fc130d8839 [term 1 LEADER]: Becoming Leader. State: Replica: e56cfa4331af483f82eee6fc130d8839, State: Running, Role: LEADER
I20260924 21:16:37.117717 29146 consensus_queue.cc:237] T 00000000000000000000000000000000 P e56cfa4331af483f82eee6fc130d8839 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e56cfa4331af483f82eee6fc130d8839" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34365 } }
I20260924 21:16:37.118100 29147 sys_catalog.cc:455] T 00000000000000000000000000000000 P e56cfa4331af483f82eee6fc130d8839 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e56cfa4331af483f82eee6fc130d8839" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e56cfa4331af483f82eee6fc130d8839" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34365 } } }
I20260924 21:16:37.118112 29148 sys_catalog.cc:455] T 00000000000000000000000000000000 P e56cfa4331af483f82eee6fc130d8839 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e56cfa4331af483f82eee6fc130d8839. Latest consensus state: current_term: 1 leader_uuid: "e56cfa4331af483f82eee6fc130d8839" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e56cfa4331af483f82eee6fc130d8839" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 34365 } } }
I20260924 21:16:37.118278 29148 sys_catalog.cc:458] T 00000000000000000000000000000000 P e56cfa4331af483f82eee6fc130d8839 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:37.118475 29155 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:16:37.118664 29147 sys_catalog.cc:458] T 00000000000000000000000000000000 P e56cfa4331af483f82eee6fc130d8839 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:37.119015 29155 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:16:37.120054 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 29075
I20260924 21:16:37.120163 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:16:37.120335 6986 external_mini_cluster.cc:1428] Setting key 77652c7dbec70d65f7fc26cec312d5ea
I20260924 21:16:37.120890 29155 catalog_manager.cc:1347] Generated new cluster ID: 442c1435ac4a4fc2833733788c46d51e
I20260924 21:16:37.120940 29155 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:16:37.131938 29166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:37.120754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54771 (local address 127.6.210.190:34365)
0924 21:16:37.120898 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:37.120907 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:37.130508 (+ 9601us) server_negotiation.cc:407] Connection header received
0924 21:16:37.130634 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:37.130639 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:37.130663 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:37.130799 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:37.131476 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:37.131480 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:37.131480 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:37.131484 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:37.131533 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:37.131544 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:37.131614 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:37.131684 (+ 70us) server_negotiation.cc:299] Negotiation successful
0924 21:16:37.131730 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":75,"threads_started":1}
I20260924 21:16:37.132941 6986 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:16:37.139390 29166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:37.134590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33027 (local address 127.6.210.190:34365)
0924 21:16:37.134626 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:37.134628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:37.138437 (+ 3809us) server_negotiation.cc:407] Connection header received
0924 21:16:37.138512 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:37.138512 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:37.138518 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:37.138582 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:37.139181 (+ 599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:37.139182 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:37.139182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:16:37.139186 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:37.139200 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:16:37.139204 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:37.139250 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:37.139302 (+ 52us) server_negotiation.cc:299] Negotiation successful
0924 21:16:37.139318 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 21:16:37.154947 29155 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:16:37.155128 29155 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:16:37.160244 29155 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e56cfa4331af483f82eee6fc130d8839: Generated new TSK 0
I20260924 21:16:37.160429 29155 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:16:37.166692 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 29075
2026-09-24T21:16:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1341 ms)
[----------] 18 tests from AuthTokenIssuingTest (24185 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:16:37 dist-test-slave-q276 krb5kdc[29174](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:16:37 dist-test-slave-q276 krb5kdc[29174](info): set up 2 sockets
Sep 24 21:16:37 dist-test-slave-q276 krb5kdc[29174](info): commencing operation
krb5kdc: starting...
W20260924 21:16:38.234515 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:16:38 dist-test-slave-q276 krb5kdc[29174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:16:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:16:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.210.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
I20260924 21:16:38.353924 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:36301
--webserver_interface=127.6.210.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:44479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:36301,127.6.210.189:38353,127.6.210.188:34797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:38.434914 29187 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 21:16:38.435128 29187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:38.435146 29187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:38.435166 29187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:38.436228 29187 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:38.436260 29187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:38.436275 29187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:38.436286 29187 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:38.436298 29187 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:38.437712 29187 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:44479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:36301,127.6.210.189:38353,127.6.210.188:34797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:36301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:38.437949 29187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:38.438187 29187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:38.443190 29192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:38.443212 29195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:38.443320 29187 server_base.cc:1034] running on GCE node
W20260924 21:16:38.443166 29193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:38.443660 29187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:38.443989 29187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:38.445153 29187 hybrid_clock.cc:648] HybridClock initialized: now 1790284598445133 us; error 38 us; skew 500 ppm
Sep 24 21:16:38 dist-test-slave-q276 krb5kdc[29174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284598, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:38.447860 29187 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:16:38.448330 29187 webserver.cc:460] Webserver started at http://127.6.210.190:39153/ using document root <none> and password file <none>
I20260924 21:16:38.448557 29187 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:38.448623 29187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:38.448784 29187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:38.449862 29187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "dda956990bb942c393e67fd846d750b7"
format_stamp: "Formatted at 2026-09-24 21:16:38 on dist-test-slave-q276"
server_key: "80a506d370dd1a9bff617c3dd4198e0d"
server_key_iv: "29b7351cae56c2ab8b6279f68e52bcf5"
server_key_version: "encryptionkey@0"
I20260924 21:16:38.450232 29187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "dda956990bb942c393e67fd846d750b7"
format_stamp: "Formatted at 2026-09-24 21:16:38 on dist-test-slave-q276"
server_key: "80a506d370dd1a9bff617c3dd4198e0d"
server_key_iv: "29b7351cae56c2ab8b6279f68e52bcf5"
server_key_version: "encryptionkey@0"
I20260924 21:16:38.451658 29187 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:38.452503 29202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:38.452672 29187 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:38.452765 29187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "dda956990bb942c393e67fd846d750b7"
format_stamp: "Formatted at 2026-09-24 21:16:38 on dist-test-slave-q276"
server_key: "80a506d370dd1a9bff617c3dd4198e0d"
server_key_iv: "29b7351cae56c2ab8b6279f68e52bcf5"
server_key_version: "encryptionkey@0"
I20260924 21:16:38.452848 29187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:38.471381 29187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:38.473557 29187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:38.473747 29187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:38.477977 29187 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:36301
I20260924 21:16:38.478016 29254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:36301 every 8 connection(s)
I20260924 21:16:38.478437 29187 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:16:38.478585 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 29187
I20260924 21:16:38.478717 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:16:38.478930 6986 external_mini_cluster.cc:1428] Setting key aa8f2cf95af730b1d54b5617fe33a427
I20260924 21:16:38.479592 29255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:38.481262 29255 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36301 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.6.210.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.189@KRBTEST.COM" created.
Sep 24 21:16:38 dist-test-slave-q276 krb5kdc[29174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284598, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:16:38.494964 29258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.482032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36301 (local address 127.0.0.1:49052)
0924 21:16:38.482207 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:38.482255 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:38.482763 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:38.483059 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:38.483062 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:38.483340 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:38.483574 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:38.483580 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.484593 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.484594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:38.485772 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:38.485775 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.486283 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.486284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:38.486364 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:38.486895 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:38.486906 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:38.491259 (+ 4353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:38.493395 (+ 2136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:38.493401 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:38.493404 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:38.493593 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:38.493872 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:38.493873 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:38.493874 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:38.493975 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:38.494481 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:38.494487 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:38.494547 (+ 60us) client_negotiation.cc:769] Sending connection context
0924 21:16:38.494619 (+ 72us) client_negotiation.cc:240] Negotiation successful
0924 21:16:38.494751 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":79,"threads_started":1}
I20260924 21:16:38.495059 29259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.482092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49052 (local address 127.6.210.190:36301)
0924 21:16:38.482295 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:38.482301 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:38.482408 (+ 107us) server_negotiation.cc:407] Connection header received
0924 21:16:38.482934 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:38.482937 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:38.482962 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:38.483054 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:38.483684 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.484500 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.485853 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.486157 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.486241 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:38.491352 (+ 5111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:38.491360 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:38.491360 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:38.491367 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:38.493317 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:38.493675 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:38.493678 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:38.493679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:38.493792 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:38.494040 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:38.494041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:38.494042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:38.494402 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:38.494487 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:38.494682 (+ 195us) server_negotiation.cc:299] Negotiation successful
0924 21:16:38.494842 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":151,"thread_start_us":98,"threads_started":1}
I20260924 21:16:38.495815 29255 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.210.189" port: 38353 } has no permanent_uuid. Determining permanent_uuid...
I20260924 21:16:38.496151 29258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.496059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:38353 (local address 127.0.0.1:40966)
0924 21:16:38.496092 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:38.496133 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.189:38353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260924 21:16:38.496296 29205 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.210.189:38353: connect: Connection refused (error 111)
I20260924 21:16:38.496713 29258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.496631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:38353 (local address 127.0.0.1:40982)
0924 21:16:38.496674 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:38.496698 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.189:38353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260924 21:16:38.496830 29255 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.210.189:38353: Network error: Client connection negotiation failed: client connection to 127.6.210.189:38353: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.189.keytab.
Entry for principal kudu/127.6.210.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.189.keytab.
I20260924 21:16:38.506976 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.189:38353
--webserver_interface=127.6.210.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.189.keytab
--principal=kudu/127.6.210.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:44479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:36301,127.6.210.189:38353,127.6.210.188:34797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:16:38.546098 29255 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.210.189" port: 38353 } attempt: 1
I20260924 21:16:38.546638 29258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.546495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:38353 (local address 127.0.0.1:40986)
0924 21:16:38.546555 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:38.546604 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.189:38353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260924 21:16:38.547065 29258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.546979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:38353 (local address 127.0.0.1:40998)
0924 21:16:38.547018 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:38.547045 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.189:38353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260924 21:16:38.547160 29255 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.210.189:38353: Network error: Client connection negotiation failed: client connection to 127.6.210.189:38353: connect: Connection refused (error 111)
W20260924 21:16:38.587539 29262 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 21:16:38.587746 29262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:38.587764 29262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:38.587783 29262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:38.588847 29262 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:38.588879 29262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:38.588893 29262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:38.588905 29262 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:38.588917 29262 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:38.590374 29262 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:44479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:36301,127.6.210.189:38353,127.6.210.188:34797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.189.keytab
--principal=kudu/127.6.210.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.189:38353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.210.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:38.590565 29262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:38.590723 29262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:38.595434 29270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:38.595476 29268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:38.595485 29267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:38.595644 29262 server_base.cc:1034] running on GCE node
I20260924 21:16:38.595909 29262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:38.596154 29262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:38.597288 29262 hybrid_clock.cc:648] HybridClock initialized: now 1790284598597269 us; error 34 us; skew 500 ppm
Sep 24 21:16:38 dist-test-slave-q276 krb5kdc[29174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284598, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:38.599820 29262 init.cc:377] Logged in from keytab as kudu/127.6.210.189@KRBTEST.COM (short username kudu)
I20260924 21:16:38.600270 29262 webserver.cc:460] Webserver started at http://127.6.210.189:34529/ using document root <none> and password file <none>
I20260924 21:16:38.600492 29262 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:38.600538 29262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:38.600657 29262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:38.601677 29262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-1/data/instance:
uuid: "6b2043fb160a4b68b6f201abd87baa27"
format_stamp: "Formatted at 2026-09-24 21:16:38 on dist-test-slave-q276"
server_key: "126a50054e80b5d426ba6d34f9ef5874"
server_key_iv: "1c9944cf1ded89ff25184347c4d1f069"
server_key_version: "encryptionkey@0"
I20260924 21:16:38.602021 29262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-1/wal/instance:
uuid: "6b2043fb160a4b68b6f201abd87baa27"
format_stamp: "Formatted at 2026-09-24 21:16:38 on dist-test-slave-q276"
server_key: "126a50054e80b5d426ba6d34f9ef5874"
server_key_iv: "1c9944cf1ded89ff25184347c4d1f069"
server_key_version: "encryptionkey@0"
I20260924 21:16:38.603381 29262 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:16:38.604123 29277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:38.604290 29262 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:16:38.604386 29262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-1/wal
uuid: "6b2043fb160a4b68b6f201abd87baa27"
format_stamp: "Formatted at 2026-09-24 21:16:38 on dist-test-slave-q276"
server_key: "126a50054e80b5d426ba6d34f9ef5874"
server_key_iv: "1c9944cf1ded89ff25184347c4d1f069"
server_key_version: "encryptionkey@0"
I20260924 21:16:38.604455 29262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:38.615298 29255 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.210.189" port: 38353 } attempt: 2
I20260924 21:16:38.615787 29258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.615630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:38353 (local address 127.0.0.1:41010)
0924 21:16:38.615697 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:38.615749 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.189:38353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260924 21:16:38.616146 29258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.616066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:38353 (local address 127.0.0.1:41026)
0924 21:16:38.616104 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:38.616129 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.189:38353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260924 21:16:38.616264 29255 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.210.189:38353: Network error: Client connection negotiation failed: client connection to 127.6.210.189:38353: connect: Connection refused (error 111)
I20260924 21:16:38.621106 29262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:38.623332 29262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:38.623484 29262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:38.627470 29262 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.189:38353
I20260924 21:16:38.627547 29329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.189:38353 every 8 connection(s)
I20260924 21:16:38.627856 29262 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-1/data/info.pb
I20260924 21:16:38.628809 29330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:38.630429 29330 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36301 } has no permanent_uuid. Determining permanent_uuid...
I20260924 21:16:38.631546 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 29262
I20260924 21:16:38.631671 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-1/wal/instance
I20260924 21:16:38.631860 6986 external_mini_cluster.cc:1428] Setting key 38407a2f64aa9ffe0c90471ed3c5725e
Sep 24 21:16:38 dist-test-slave-q276 krb5kdc[29174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284598, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.189@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.210.188@KRBTEST.COM; defaulting to no policy
I20260924 21:16:38.640785 29259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.631061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49064 (local address 127.6.210.190:36301)
0924 21:16:38.631121 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:38.631123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:38.631313 (+ 190us) server_negotiation.cc:407] Connection header received
0924 21:16:38.631915 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:38.631916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:38.631934 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:38.632001 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:38.632585 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.633283 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.634597 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.634857 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.634936 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:38.637470 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:38.637471 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:38.637472 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:38.637480 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:38.639038 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:38.639598 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:38.639599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:38.639600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:38.639661 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:38.640013 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:38.640015 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:38.640015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:38.640145 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:38.640221 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:38.640580 (+ 359us) server_negotiation.cc:299] Negotiation successful
0924 21:16:38.640671 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 21:16:38.640985 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.631048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36301 (local address 127.0.0.1:49064)
0924 21:16:38.631204 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:38.631239 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:38.631789 (+ 550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:38.632068 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:38.632071 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:38.632304 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:38.632502 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:38.632506 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.633377 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.633378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:38.634499 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:38.634503 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.634965 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.634966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:38.635137 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:38.635647 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:38.635657 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:38.637364 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:38.639141 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:38.639149 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:38.639152 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:38.639517 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:38.639816 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:38.639817 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:38.639818 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:38.639940 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:38.640266 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:38.640273 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:38.640438 (+ 165us) client_negotiation.cc:769] Sending connection context
0924 21:16:38.640556 (+ 118us) client_negotiation.cc:240] Negotiation successful
0924 21:16:38.640726 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":83,"threads_started":1}
I20260924 21:16:38.641641 29330 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.210.189" port: 38353 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.188@KRBTEST.COM" created.
Sep 24 21:16:38 dist-test-slave-q276 krb5kdc[29174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284598, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.189@KRBTEST.COM for kudu/127.6.210.189@KRBTEST.COM
I20260924 21:16:38.650418 29334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.641988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41032 (local address 127.6.210.189:38353)
0924 21:16:38.642154 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:38.642159 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:38.642181 (+ 22us) server_negotiation.cc:407] Connection header received
0924 21:16:38.642198 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:38.642199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:38.642210 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:38.642342 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:38.642644 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.643879 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.644894 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.645458 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.645535 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:38.647700 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:38.647706 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:38.647706 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:38.647712 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:38.649177 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:38.649595 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:38.649597 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:38.649598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:38.649647 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:38.649852 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:38.649853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:38.649853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:38.649988 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:38.650049 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:38.650226 (+ 177us) server_negotiation.cc:299] Negotiation successful
0924 21:16:38.650342 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":80,"threads_started":1}
I20260924 21:16:38.650540 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.641953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:38353 (local address 127.0.0.1:41032)
0924 21:16:38.641987 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:38.642004 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:38.642042 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:38.642350 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:38.642351 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:38.642518 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:38.642568 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:38.642570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.643982 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.643982 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:38.644807 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:38.644810 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.645541 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.645541 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:38.645594 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:38.645988 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:38.645992 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:38.647612 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:38.649265 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:38.649266 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:38.649266 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:38.649508 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:38.649709 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:38.649710 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:38.649711 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:38.649784 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:38.650058 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:38.650059 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:38.650108 (+ 49us) client_negotiation.cc:769] Sending connection context
0924 21:16:38.650200 (+ 92us) client_negotiation.cc:240] Negotiation successful
0924 21:16:38.650429 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18,"mutex_wait_us":104}
I20260924 21:16:38.651170 29330 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.210.188" port: 34797 } has no permanent_uuid. Determining permanent_uuid...
I20260924 21:16:38.651523 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.651401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:34797 (local address 127.0.0.1:35382)
0924 21:16:38.651449 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:38.651494 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.188:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260924 21:16:38.651655 29280 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.210.188:34797: connect: Connection refused (error 111)
I20260924 21:16:38.652081 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.651992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:34797 (local address 127.0.0.1:35392)
0924 21:16:38.652038 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:38.652064 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.188:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260924 21:16:38.652195 29330 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.210.188:34797: Network error: Client connection negotiation failed: client connection to 127.6.210.188:34797: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.188.keytab.
Entry for principal kudu/127.6.210.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.188.keytab.
I20260924 21:16:38.661341 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.188:34797
--webserver_interface=127.6.210.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.188.keytab
--principal=kudu/127.6.210.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:44479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:36301,127.6.210.189:38353,127.6.210.188:34797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:16:38.701522 29330 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.210.188" port: 34797 } attempt: 1
I20260924 21:16:38.702069 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.701914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:34797 (local address 127.0.0.1:35408)
0924 21:16:38.701979 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:38.702030 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.188:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260924 21:16:38.702462 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.702374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:34797 (local address 127.0.0.1:35414)
0924 21:16:38.702416 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:38.702442 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.188:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260924 21:16:38.702576 29330 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.210.188:34797: Network error: Client connection negotiation failed: client connection to 127.6.210.188:34797: connect: Connection refused (error 111)
I20260924 21:16:38.707430 29255 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.210.189" port: 38353 } attempt: 3
Sep 24 21:16:38 dist-test-slave-q276 krb5kdc[29174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284598, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for kudu/127.6.210.189@KRBTEST.COM
I20260924 21:16:38.714658 29258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.707844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:38353 (local address 127.0.0.1:41036)
0924 21:16:38.707922 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:38.707944 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:38.708001 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:38.708172 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:38.708173 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:38.708329 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:38.708395 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:38.708397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.709035 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.709036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:38.709554 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:38.709556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.709917 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.709918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:38.709993 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:38.710496 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:38.710501 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:38.711932 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:38.713521 (+ 1589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:38.713522 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:38.713522 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:38.713696 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:38.713984 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:38.713985 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:38.713985 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:38.714048 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:38.714329 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:38.714330 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:38.714377 (+ 47us) client_negotiation.cc:769] Sending connection context
0924 21:16:38.714458 (+ 81us) client_negotiation.cc:240] Negotiation successful
0924 21:16:38.714561 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260924 21:16:38.714658 29334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.707913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41036 (local address 127.6.210.189:38353)
0924 21:16:38.707968 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:38.707970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:38.707986 (+ 16us) server_negotiation.cc:407] Connection header received
0924 21:16:38.708081 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:38.708082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:38.708094 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:38.708153 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:38.708470 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.708959 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.709632 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.709845 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.709918 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:38.712035 (+ 2117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:38.712036 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:38.712037 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:38.712045 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:38.713433 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:38.713855 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:38.713857 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:38.713857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:38.713915 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:38.714108 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:38.714109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:38.714110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:38.714224 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:38.714336 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:38.714484 (+ 148us) server_negotiation.cc:299] Negotiation successful
0924 21:16:38.714562 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 21:16:38.715217 29255 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.210.188" port: 34797 } has no permanent_uuid. Determining permanent_uuid...
I20260924 21:16:38.715543 29258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.715452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:34797 (local address 127.0.0.1:35422)
0924 21:16:38.715493 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:38.715523 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.188:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260924 21:16:38.715822 29258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.715764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:34797 (local address 127.0.0.1:35424)
0924 21:16:38.715786 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:38.715808 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.188:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260924 21:16:38.715916 29255 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.210.188:34797: Network error: Client connection negotiation failed: client connection to 127.6.210.188:34797: connect: Connection refused (error 111)
W20260924 21:16:38.743468 29337 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 21:16:38.743678 29337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:38.743698 29337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:38.743717 29337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:38.744809 29337 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:38.744844 29337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:38.744858 29337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:38.744870 29337 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:38.744882 29337 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:38.746306 29337 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:44479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:36301,127.6.210.189:38353,127.6.210.188:34797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.188.keytab
--principal=kudu/127.6.210.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.188:34797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.210.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:38.746497 29337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:38.746668 29337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 21:16:38.747054 29255 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.210.188" port: 34797 } attempt: 1
I20260924 21:16:38.747529 29258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.747403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:34797 (local address 127.0.0.1:35440)
0924 21:16:38.747461 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:38.747503 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.188:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260924 21:16:38.747931 29258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.747824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:34797 (local address 127.0.0.1:35452)
0924 21:16:38.747877 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:38.747907 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.188:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260924 21:16:38.748055 29255 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.210.188:34797: Network error: Client connection negotiation failed: client connection to 127.6.210.188:34797: connect: Connection refused (error 111)
W20260924 21:16:38.751569 29342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:38.751538 29343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:38.751713 29337 server_base.cc:1034] running on GCE node
W20260924 21:16:38.751560 29345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:38.752043 29337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:38.752337 29337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:38.753494 29337 hybrid_clock.cc:648] HybridClock initialized: now 1790284598753471 us; error 35 us; skew 500 ppm
Sep 24 21:16:38 dist-test-slave-q276 krb5kdc[29174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284598, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:38.755964 29337 init.cc:377] Logged in from keytab as kudu/127.6.210.188@KRBTEST.COM (short username kudu)
I20260924 21:16:38.756428 29337 webserver.cc:460] Webserver started at http://127.6.210.188:35127/ using document root <none> and password file <none>
I20260924 21:16:38.756659 29337 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:38.756721 29337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:38.756850 29337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:38.757865 29337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-2/data/instance:
uuid: "9908dc7ea9da4e319287bf39f90bb8d2"
format_stamp: "Formatted at 2026-09-24 21:16:38 on dist-test-slave-q276"
server_key: "b5f9e108a4c6e72e5881a4a7e7054567"
server_key_iv: "734913102585324bdd320aef1f3f6c3a"
server_key_version: "encryptionkey@0"
I20260924 21:16:38.758222 29337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-2/wal/instance:
uuid: "9908dc7ea9da4e319287bf39f90bb8d2"
format_stamp: "Formatted at 2026-09-24 21:16:38 on dist-test-slave-q276"
server_key: "b5f9e108a4c6e72e5881a4a7e7054567"
server_key_iv: "734913102585324bdd320aef1f3f6c3a"
server_key_version: "encryptionkey@0"
I20260924 21:16:38.759547 29337 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:16:38.760355 29352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:38.760530 29337 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:16:38.760619 29337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-2/wal
uuid: "9908dc7ea9da4e319287bf39f90bb8d2"
format_stamp: "Formatted at 2026-09-24 21:16:38 on dist-test-slave-q276"
server_key: "b5f9e108a4c6e72e5881a4a7e7054567"
server_key_iv: "734913102585324bdd320aef1f3f6c3a"
server_key_version: "encryptionkey@0"
I20260924 21:16:38.760697 29337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:38.770723 29330 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.210.188" port: 34797 } attempt: 2
I20260924 21:16:38.771150 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.771015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:34797 (local address 127.0.0.1:35468)
0924 21:16:38.771073 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:38.771115 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.188:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260924 21:16:38.771516 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.771444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:34797 (local address 127.0.0.1:35472)
0924 21:16:38.771479 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:38.771501 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.188:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260924 21:16:38.771631 29330 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.210.188:34797: Network error: Client connection negotiation failed: client connection to 127.6.210.188:34797: connect: Connection refused (error 111)
I20260924 21:16:38.773993 29337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:38.776068 29337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:38.776206 29337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:38.780256 29337 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.188:34797
I20260924 21:16:38.780318 29404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.188:34797 every 8 connection(s)
I20260924 21:16:38.780643 29337 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-2/data/info.pb
I20260924 21:16:38.781757 29405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:38.783566 29405 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36301 } has no permanent_uuid. Determining permanent_uuid...
I20260924 21:16:38.785820 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 29337
I20260924 21:16:38.785964 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284333163253-6986-0/minicluster-data/master-2/wal/instance
I20260924 21:16:38.786134 6986 external_mini_cluster.cc:1428] Setting key 9fd3cb228eeccd0472ab8e8dcd2f6f4d
Sep 24 21:16:38 dist-test-slave-q276 krb5kdc[29174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284598, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.188@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
Sep 24 21:16:38 dist-test-slave-q276 krb5kdc[29174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.188@KRBTEST.COM
I20260924 21:16:38.793356 29259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.784279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49072 (local address 127.6.210.190:36301)
0924 21:16:38.784346 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:38.784348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:38.784527 (+ 179us) server_negotiation.cc:407] Connection header received
0924 21:16:38.785013 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:38.785014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:38.785026 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:38.785096 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:38.785709 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.786403 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.787589 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.787901 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.787971 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:38.790181 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:38.790182 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:38.790182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:38.790197 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:38.791652 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:38.792148 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:38.792149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:38.792150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:38.792215 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:38.792683 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:38.792684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:38.792685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:38.792818 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:38.792890 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:38.793153 (+ 263us) server_negotiation.cc:299] Negotiation successful
0924 21:16:38.793247 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 21:16:38.793681 29407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.784253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:36301 (local address 127.0.0.1:49072)
0924 21:16:38.784402 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:38.784442 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:38.784875 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:38.785163 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:38.785166 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:38.785405 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:38.785589 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:38.785593 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.786531 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.786532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:38.787497 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:38.787500 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.788017 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.788019 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:38.788126 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:38.788649 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:38.788656 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:38.790091 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:38.791766 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:38.791772 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:38.791774 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:38.792073 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:38.792495 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:38.792496 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:38.792497 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:38.792616 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:38.792887 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:38.792893 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:38.793025 (+ 132us) client_negotiation.cc:769] Sending connection context
0924 21:16:38.793132 (+ 107us) client_negotiation.cc:240] Negotiation successful
0924 21:16:38.793271 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":74,"threads_started":1}
I20260924 21:16:38.794247 29409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.786494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35478 (local address 127.6.210.188:34797)
0924 21:16:38.786694 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:38.786698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:38.786721 (+ 23us) server_negotiation.cc:407] Connection header received
0924 21:16:38.786753 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:38.786755 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:38.786780 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:38.786863 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:38.787438 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.788328 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.789050 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.789351 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.789441 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:38.791590 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:38.791596 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:38.791596 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:38.791603 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:38.792967 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:38.793306 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:38.793309 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:38.793310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:38.793367 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:38.793650 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:38.793651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:38.793651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:38.793837 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:38.793973 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:38.794074 (+ 101us) server_negotiation.cc:299] Negotiation successful
0924 21:16:38.794148 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":141,"threads_started":1}
I20260924 21:16:38.794251 29405 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.210.189" port: 38353 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 21:16:38 dist-test-slave-q276 krb5kdc[29174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284598, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.188@KRBTEST.COM for kudu/127.6.210.189@KRBTEST.COM
Sep 24 21:16:38 dist-test-slave-q276 krb5kdc[29174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:16:38.801633 29407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.794592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:38353 (local address 127.0.0.1:41052)
0924 21:16:38.794640 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:38.794650 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:38.794702 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:38.794830 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:38.794830 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:38.794971 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:38.795034 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:38.795036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.795885 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.795886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:38.796576 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:38.796578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.797072 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.797073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:38.797148 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:38.797511 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:38.797516 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:38.798815 (+ 1299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:38.800591 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:38.800592 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:38.800593 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:38.800803 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:38.801043 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:38.801044 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:38.801044 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:38.801117 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:38.801417 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:38.801417 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:38.801452 (+ 35us) client_negotiation.cc:769] Sending connection context
0924 21:16:38.801506 (+ 54us) client_negotiation.cc:240] Negotiation successful
0924 21:16:38.801574 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260924 21:16:38.801710 29334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.794680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41052 (local address 127.6.210.189:38353)
0924 21:16:38.794724 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:38.794726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:38.794734 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:16:38.794763 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:38.794764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:38.794774 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:38.794818 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:38.795112 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.795803 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.796646 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.796990 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.797064 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:38.798905 (+ 1841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:38.798907 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:38.798907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:38.798914 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:38.800504 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:38.800892 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:38.800893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:38.800894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:38.800968 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:38.801178 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:38.801179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:38.801179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:38.801336 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:38.801413 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:38.801549 (+ 136us) server_negotiation.cc:299] Negotiation successful
0924 21:16:38.801625 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 21:16:38.802053 29405 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.210.188" port: 34797 } has no permanent_uuid. Determining permanent_uuid...
I20260924 21:16:38.803216 29259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.794939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49080 (local address 127.6.210.190:36301)
0924 21:16:38.794975 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:38.794977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:38.794984 (+ 7us) server_negotiation.cc:407] Connection header received
0924 21:16:38.795002 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:38.795003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:38.795012 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:38.795083 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:38.795642 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.796329 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.797154 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.797446 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.797513 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:38.800267 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:38.800268 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:38.800269 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:38.800276 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:38.802020 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:38.802481 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:38.802482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:38.802482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:38.802529 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:38.802735 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:38.802736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:38.802736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:38.802847 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:38.803012 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:38.803048 (+ 36us) server_negotiation.cc:299] Negotiation successful
0924 21:16:38.803120 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Sep 24 21:16:38 dist-test-slave-q276 krb5kdc[29174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284598, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.188@KRBTEST.COM for kudu/127.6.210.188@KRBTEST.COM
Sep 24 21:16:38 dist-test-slave-q276 krb5kdc[29174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.189@KRBTEST.COM
I20260924 21:16:38.809075 29409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.802391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35494 (local address 127.6.210.188:34797)
0924 21:16:38.802424 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:38.802426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:38.802437 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:16:38.802510 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:38.802511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:38.802519 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:38.802560 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:38.802788 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.803284 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.804070 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.804279 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.804352 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:38.805928 (+ 1576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:38.805929 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:38.805930 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:38.805936 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:38.807405 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:38.808258 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:38.808260 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:38.808260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:38.808308 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:38.808494 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:38.808495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:38.808495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:38.808616 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:38.808678 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:38.808862 (+ 184us) server_negotiation.cc:299] Negotiation successful
0924 21:16:38.808974 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":17}
I20260924 21:16:38.809139 29407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.802360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:34797 (local address 127.0.0.1:35494)
0924 21:16:38.802403 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:38.802412 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:38.802446 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:38.802574 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:38.802575 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:38.802691 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:38.802715 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:38.802716 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.803339 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.803340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:38.803987 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:38.803989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.804353 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.804354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:38.804408 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:38.804773 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:38.804777 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:38.805831 (+ 1054us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:38.807958 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:38.807959 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:38.807959 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:38.808181 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:38.808375 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:38.808375 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:38.808376 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:38.808429 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:38.808700 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:38.808701 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:38.808748 (+ 47us) client_negotiation.cc:769] Sending connection context
0924 21:16:38.808829 (+ 81us) client_negotiation.cc:240] Negotiation successful
0924 21:16:38.808933 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260924 21:16:38.811172 29334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.803849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41068 (local address 127.6.210.189:38353)
0924 21:16:38.803888 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:38.803890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:38.803908 (+ 18us) server_negotiation.cc:407] Connection header received
0924 21:16:38.803997 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:38.803998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:38.804006 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:38.804061 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:38.804531 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.804973 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.805619 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.805849 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.806181 (+ 332us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:38.808377 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:38.808379 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:38.808379 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:38.808387 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:38.810073 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:38.810448 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:38.810449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:38.810449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:38.810509 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:38.810674 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:38.810675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:38.810675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:38.810799 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:38.810976 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:38.811018 (+ 42us) server_negotiation.cc:299] Negotiation successful
0924 21:16:38.811088 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 21:16:38.811156 29405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9908dc7ea9da4e319287bf39f90bb8d2: Bootstrap starting.
I20260924 21:16:38.812222 29405 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9908dc7ea9da4e319287bf39f90bb8d2: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:38.812570 29405 log.cc:824] T 00000000000000000000000000000000 P 9908dc7ea9da4e319287bf39f90bb8d2: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:38.812590 6986 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:16:38.813370 29405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9908dc7ea9da4e319287bf39f90bb8d2: No bootstrap required, opened a new log
I20260924 21:16:38.813764 29229 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "dda956990bb942c393e67fd846d750b7"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49080
I20260924 21:16:38.815035 29405 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9908dc7ea9da4e319287bf39f90bb8d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dda956990bb942c393e67fd846d750b7" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36301 } } peers { permanent_uuid: "6b2043fb160a4b68b6f201abd87baa27" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 38353 } } peers { permanent_uuid: "9908dc7ea9da4e319287bf39f90bb8d2" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 34797 } }
I20260924 21:16:38.815194 29405 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9908dc7ea9da4e319287bf39f90bb8d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9908dc7ea9da4e319287bf39f90bb8d2, State: Initialized, Role: FOLLOWER
I20260924 21:16:38.815294 29405 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9908dc7ea9da4e319287bf39f90bb8d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dda956990bb942c393e67fd846d750b7" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36301 } } peers { permanent_uuid: "6b2043fb160a4b68b6f201abd87baa27" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 38353 } } peers { permanent_uuid: "9908dc7ea9da4e319287bf39f90bb8d2" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 34797 } }
I20260924 21:16:38.815575 29411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9908dc7ea9da4e319287bf39f90bb8d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dda956990bb942c393e67fd846d750b7" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36301 } } peers { permanent_uuid: "6b2043fb160a4b68b6f201abd87baa27" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 38353 } } peers { permanent_uuid: "9908dc7ea9da4e319287bf39f90bb8d2" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 34797 } } }
I20260924 21:16:38.815627 29405 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9908dc7ea9da4e319287bf39f90bb8d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:38.815676 29411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9908dc7ea9da4e319287bf39f90bb8d2 [sys.catalog]: This master's current role is: FOLLOWER
W20260924 21:16:38.817442 29422 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9908dc7ea9da4e319287bf39f90bb8d2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 21:16:38.817548 29422 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 21:16:38.817703 29337 master_runner.cc:186] Error getting master registration for 127.6.210.190:36301: OK, instance_id { permanent_uuid: "dda956990bb942c393e67fd846d750b7" instance_seqno: 1790284598476848 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 21:16:38.819782 29337 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 21:16:38.823177 29255 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.210.188" port: 34797 } attempt: 2
Sep 24 21:16:38 dist-test-slave-q276 krb5kdc[29174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284598, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for kudu/127.6.210.188@KRBTEST.COM
I20260924 21:16:38.830821 29258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.823467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:34797 (local address 127.0.0.1:35504)
0924 21:16:38.823516 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:38.823529 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:38.823580 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:38.823716 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:38.823717 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:38.823870 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:38.823911 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:38.823913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.824528 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.824530 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:38.825039 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:38.825040 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.825409 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.825410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:38.825468 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:38.825925 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:38.825931 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:38.827543 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:38.829641 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:38.829642 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:38.829642 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:38.829868 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:38.830099 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:38.830099 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:38.830100 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:38.830163 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:38.830500 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:38.830501 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:38.830548 (+ 47us) client_negotiation.cc:769] Sending connection context
0924 21:16:38.830621 (+ 73us) client_negotiation.cc:240] Negotiation successful
0924 21:16:38.830717 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260924 21:16:38.830838 29409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.823521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35504 (local address 127.6.210.188:34797)
0924 21:16:38.823579 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:38.823581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:38.823590 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:16:38.823630 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:38.823631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:38.823644 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:38.823694 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:38.823986 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.824452 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.825118 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.825324 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.825408 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:38.827640 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:38.827642 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:38.827642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:38.827649 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:38.829488 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:38.829964 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:38.829965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:38.829966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:38.830031 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:38.830226 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:38.830227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:38.830227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:38.830431 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:38.830507 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:38.830644 (+ 137us) server_negotiation.cc:299] Negotiation successful
0924 21:16:38.830732 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 21:16:38.832459 29255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dda956990bb942c393e67fd846d750b7: Bootstrap starting.
I20260924 21:16:38.833178 29255 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dda956990bb942c393e67fd846d750b7: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:38.833482 29255 log.cc:824] T 00000000000000000000000000000000 P dda956990bb942c393e67fd846d750b7: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:38.834345 29255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dda956990bb942c393e67fd846d750b7: No bootstrap required, opened a new log
I20260924 21:16:38.835703 29255 raft_consensus.cc:348] T 00000000000000000000000000000000 P dda956990bb942c393e67fd846d750b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dda956990bb942c393e67fd846d750b7" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36301 } } peers { permanent_uuid: "6b2043fb160a4b68b6f201abd87baa27" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 38353 } } peers { permanent_uuid: "9908dc7ea9da4e319287bf39f90bb8d2" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 34797 } }
I20260924 21:16:38.835848 29255 raft_consensus.cc:729] T 00000000000000000000000000000000 P dda956990bb942c393e67fd846d750b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dda956990bb942c393e67fd846d750b7, State: Initialized, Role: FOLLOWER
I20260924 21:16:38.835940 29255 consensus_queue.cc:260] T 00000000000000000000000000000000 P dda956990bb942c393e67fd846d750b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dda956990bb942c393e67fd846d750b7" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36301 } } peers { permanent_uuid: "6b2043fb160a4b68b6f201abd87baa27" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 38353 } } peers { permanent_uuid: "9908dc7ea9da4e319287bf39f90bb8d2" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 34797 } }
I20260924 21:16:38.836256 29255 sys_catalog.cc:564] T 00000000000000000000000000000000 P dda956990bb942c393e67fd846d750b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:38.836249 29425 sys_catalog.cc:455] T 00000000000000000000000000000000 P dda956990bb942c393e67fd846d750b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dda956990bb942c393e67fd846d750b7" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36301 } } peers { permanent_uuid: "6b2043fb160a4b68b6f201abd87baa27" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 38353 } } peers { permanent_uuid: "9908dc7ea9da4e319287bf39f90bb8d2" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 34797 } } }
I20260924 21:16:38.836320 29229 raft_consensus.cc:482] T 00000000000000000000000000000000 P dda956990bb942c393e67fd846d750b7 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 21:16:38.836444 29229 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dda956990bb942c393e67fd846d750b7 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:38.837235 29229 raft_consensus.cc:504] T 00000000000000000000000000000000 P dda956990bb942c393e67fd846d750b7 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dda956990bb942c393e67fd846d750b7" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36301 } } peers { permanent_uuid: "6b2043fb160a4b68b6f201abd87baa27" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 38353 } } peers { permanent_uuid: "9908dc7ea9da4e319287bf39f90bb8d2" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 34797 } }
I20260924 21:16:38.837491 29229 leader_election.cc:288] T 00000000000000000000000000000000 P dda956990bb942c393e67fd846d750b7 [CANDIDATE]: Term 1 election: Requested vote from peers 6b2043fb160a4b68b6f201abd87baa27 (127.6.210.189:38353), 9908dc7ea9da4e319287bf39f90bb8d2 (127.6.210.188:34797)
I20260924 21:16:38.838759 29379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dda956990bb942c393e67fd846d750b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9908dc7ea9da4e319287bf39f90bb8d2"
I20260924 21:16:38.838922 29379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9908dc7ea9da4e319287bf39f90bb8d2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:38.839876 29379 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9908dc7ea9da4e319287bf39f90bb8d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dda956990bb942c393e67fd846d750b7 in term 1.
I20260924 21:16:38.840138 29205 leader_election.cc:302] T 00000000000000000000000000000000 P dda956990bb942c393e67fd846d750b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9908dc7ea9da4e319287bf39f90bb8d2, dda956990bb942c393e67fd846d750b7; no voters:
I20260924 21:16:38.840121 29425 sys_catalog.cc:458] T 00000000000000000000000000000000 P dda956990bb942c393e67fd846d750b7 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:16:38.840451 29425 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dda956990bb942c393e67fd846d750b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:38.840516 29425 raft_consensus.cc:686] T 00000000000000000000000000000000 P dda956990bb942c393e67fd846d750b7 [term 1 LEADER]: Becoming Leader. State: Replica: dda956990bb942c393e67fd846d750b7, State: Running, Role: LEADER
I20260924 21:16:38.840657 29425 consensus_queue.cc:237] T 00000000000000000000000000000000 P dda956990bb942c393e67fd846d750b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dda956990bb942c393e67fd846d750b7" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36301 } } peers { permanent_uuid: "6b2043fb160a4b68b6f201abd87baa27" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 38353 } } peers { permanent_uuid: "9908dc7ea9da4e319287bf39f90bb8d2" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 34797 } }
I20260924 21:16:38.841120 29425 sys_catalog.cc:455] T 00000000000000000000000000000000 P dda956990bb942c393e67fd846d750b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dda956990bb942c393e67fd846d750b7. Latest consensus state: current_term: 1 leader_uuid: "dda956990bb942c393e67fd846d750b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dda956990bb942c393e67fd846d750b7" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36301 } } peers { permanent_uuid: "6b2043fb160a4b68b6f201abd87baa27" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 38353 } } peers { permanent_uuid: "9908dc7ea9da4e319287bf39f90bb8d2" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 34797 } } }
I20260924 21:16:38.841228 29425 sys_catalog.cc:458] T 00000000000000000000000000000000 P dda956990bb942c393e67fd846d750b7 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:38.838778 29187 master_runner.cc:186] Error getting master registration for 127.6.210.189:38353: OK, instance_id { permanent_uuid: "6b2043fb160a4b68b6f201abd87baa27" instance_seqno: 1790284598626368 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 21:16:38.841509 29187 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 21:16:38.842231 29444 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:16:38.842811 29444 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:16:38.846868 29379 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9908dc7ea9da4e319287bf39f90bb8d2 [term 1 FOLLOWER]: Refusing update from remote peer dda956990bb942c393e67fd846d750b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:16:38.847232 29442 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dda956990bb942c393e67fd846d750b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9908dc7ea9da4e319287bf39f90bb8d2" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 34797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:16:38.848652 29411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9908dc7ea9da4e319287bf39f90bb8d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dda956990bb942c393e67fd846d750b7. Latest consensus state: current_term: 1 leader_uuid: "dda956990bb942c393e67fd846d750b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dda956990bb942c393e67fd846d750b7" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36301 } } peers { permanent_uuid: "6b2043fb160a4b68b6f201abd87baa27" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 38353 } } peers { permanent_uuid: "9908dc7ea9da4e319287bf39f90bb8d2" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 34797 } } }
I20260924 21:16:38.848773 29411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9908dc7ea9da4e319287bf39f90bb8d2 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:16:38.851044 29334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.841580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41072 (local address 127.6.210.189:38353)
0924 21:16:38.841628 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:38.841630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:38.842238 (+ 608us) server_negotiation.cc:407] Connection header received
0924 21:16:38.842445 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:38.842447 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:38.842460 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:38.842497 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:38.843394 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.844101 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.845155 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.845459 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.845514 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:38.847632 (+ 2118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:38.847634 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:38.847634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:38.847642 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:38.849496 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:38.850026 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:38.850028 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:38.850028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:38.850098 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:38.850379 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:38.850380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:38.850380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:38.850523 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:38.850618 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:38.850857 (+ 239us) server_negotiation.cc:299] Negotiation successful
0924 21:16:38.850942 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 21:16:38.851991 29259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.838828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49090 (local address 127.6.210.190:36301)
0924 21:16:38.838874 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:38.838876 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:38.842019 (+ 3143us) server_negotiation.cc:407] Connection header received
0924 21:16:38.842178 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:38.842179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:38.842202 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:38.842232 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:38.843413 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.844148 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.845666 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.846018 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.846065 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:38.848611 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:38.848612 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:38.848624 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:38.848630 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:38.850397 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:38.850969 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:38.850970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:38.850970 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:38.851035 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:38.851316 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:38.851317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:38.851317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:38.851447 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:38.851513 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:38.851817 (+ 304us) server_negotiation.cc:299] Negotiation successful
0924 21:16:38.851901 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 21:16:38.856675 29442 sys_catalog.cc:455] T 00000000000000000000000000000000 P dda956990bb942c393e67fd846d750b7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dda956990bb942c393e67fd846d750b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dda956990bb942c393e67fd846d750b7" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36301 } } peers { permanent_uuid: "6b2043fb160a4b68b6f201abd87baa27" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 38353 } } peers { permanent_uuid: "9908dc7ea9da4e319287bf39f90bb8d2" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 34797 } } }
I20260924 21:16:38.856914 29442 sys_catalog.cc:458] T 00000000000000000000000000000000 P dda956990bb942c393e67fd846d750b7 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:38.857084 29411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9908dc7ea9da4e319287bf39f90bb8d2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dda956990bb942c393e67fd846d750b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dda956990bb942c393e67fd846d750b7" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36301 } } peers { permanent_uuid: "6b2043fb160a4b68b6f201abd87baa27" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 38353 } } peers { permanent_uuid: "9908dc7ea9da4e319287bf39f90bb8d2" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 34797 } } }
I20260924 21:16:38.857180 29411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9908dc7ea9da4e319287bf39f90bb8d2 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:16:38.857275 29448 mvcc.cc:205] Tried to move back new op lower bound from 7333005716872790016 to 7333005716852137984. Current Snapshot: MvccSnapshot[applied={T|T < 7333005716872790016}]
I20260924 21:16:38.857375 29449 mvcc.cc:205] Tried to move back new op lower bound from 7333005716872790016 to 7333005716852137984. Current Snapshot: MvccSnapshot[applied={T|T < 7333005716872790016}]
I20260924 21:16:38.859551 29409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.841996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35510 (local address 127.6.210.188:34797)
0924 21:16:38.842048 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:38.842050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:38.851576 (+ 9526us) server_negotiation.cc:407] Connection header received
0924 21:16:38.851692 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:38.851693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:38.851706 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:38.851741 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:38.852671 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.853341 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.854322 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.854631 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.854699 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:38.856368 (+ 1669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:38.856370 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:38.856370 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:38.856377 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:38.858163 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:38.858657 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:38.858658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:38.858658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:38.858716 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:38.858941 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:38.858942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:38.858942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:38.859071 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:38.859161 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:38.859380 (+ 219us) server_negotiation.cc:299] Negotiation successful
0924 21:16:38.859465 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 21:16:38.862721 29444 catalog_manager.cc:1347] Generated new cluster ID: 0411d07630f343a39ecc6f30f34b218d
I20260924 21:16:38.862785 29444 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:16:38.870293 29330 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.210.188" port: 34797 } attempt: 3
Sep 24 21:16:38 dist-test-slave-q276 krb5kdc[29174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284598, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.189@KRBTEST.COM for kudu/127.6.210.188@KRBTEST.COM
I20260924 21:16:38.880227 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.870589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:34797 (local address 127.0.0.1:35526)
0924 21:16:38.870681 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:38.870695 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:38.870742 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:38.870999 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:38.870999 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:38.871205 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:38.871259 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:38.871261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.872213 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.872215 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:38.872997 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:38.873000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.873489 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.873490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:38.873546 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:38.874135 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:38.874140 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:38.875661 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:38.878839 (+ 3178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:38.878840 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:38.878841 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:38.879064 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:38.879397 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:38.879397 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:38.879398 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:38.879471 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:38.879928 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:38.879930 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:38.879975 (+ 45us) client_negotiation.cc:769] Sending connection context
0924 21:16:38.880023 (+ 48us) client_negotiation.cc:240] Negotiation successful
0924 21:16:38.880120 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260924 21:16:38.881119 29409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:38.870825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35526 (local address 127.6.210.188:34797)
0924 21:16:38.870868 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:38.870870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:38.870878 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:16:38.870901 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:38.870902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:38.870913 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:38.871337 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:38.871371 (+ 34us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.872085 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.873106 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:38.873397 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:38.875756 (+ 2359us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:38.875806 (+ 50us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:38.875807 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:38.875807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:38.875812 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:38.878709 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:38.879198 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:38.879199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:38.879200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:38.879286 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:38.879560 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:38.879561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:38.879561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:38.879804 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:38.880719 (+ 915us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:38.880822 (+ 103us) server_negotiation.cc:299] Negotiation successful
0924 21:16:38.880972 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 21:16:38.882966 29330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b2043fb160a4b68b6f201abd87baa27: Bootstrap starting.
I20260924 21:16:38.883734 29330 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b2043fb160a4b68b6f201abd87baa27: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:38.884049 29330 log.cc:824] T 00000000000000000000000000000000 P 6b2043fb160a4b68b6f201abd87baa27: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:38.886782 29444 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:16:38.887480 29444 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:16:38.890941 29330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b2043fb160a4b68b6f201abd87baa27: No bootstrap required, opened a new log
I20260924 21:16:38.892925 29330 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b2043fb160a4b68b6f201abd87baa27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dda956990bb942c393e67fd846d750b7" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36301 } } peers { permanent_uuid: "6b2043fb160a4b68b6f201abd87baa27" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 38353 } } peers { permanent_uuid: "9908dc7ea9da4e319287bf39f90bb8d2" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 34797 } }
I20260924 21:16:38.893067 29330 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b2043fb160a4b68b6f201abd87baa27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b2043fb160a4b68b6f201abd87baa27, State: Initialized, Role: FOLLOWER
I20260924 21:16:38.893148 29330 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b2043fb160a4b68b6f201abd87baa27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dda956990bb942c393e67fd846d750b7" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36301 } } peers { permanent_uuid: "6b2043fb160a4b68b6f201abd87baa27" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 38353 } } peers { permanent_uuid: "9908dc7ea9da4e319287bf39f90bb8d2" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 34797 } }
I20260924 21:16:38.893491 29330 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b2043fb160a4b68b6f201abd87baa27 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:38.893514 29304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dda956990bb942c393e67fd846d750b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6b2043fb160a4b68b6f201abd87baa27"
I20260924 21:16:38.893630 29304 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b2043fb160a4b68b6f201abd87baa27 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:38.894527 29304 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6b2043fb160a4b68b6f201abd87baa27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dda956990bb942c393e67fd846d750b7 in term 1.
I20260924 21:16:38.894730 29303 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6b2043fb160a4b68b6f201abd87baa27 [term 1 FOLLOWER]: Refusing update from remote peer dda956990bb942c393e67fd846d750b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:16:38.895287 29442 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dda956990bb942c393e67fd846d750b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b2043fb160a4b68b6f201abd87baa27" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 38353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:16:38.896790 29454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b2043fb160a4b68b6f201abd87baa27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dda956990bb942c393e67fd846d750b7" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36301 } } peers { permanent_uuid: "6b2043fb160a4b68b6f201abd87baa27" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 38353 } } peers { permanent_uuid: "9908dc7ea9da4e319287bf39f90bb8d2" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 34797 } } }
I20260924 21:16:38.896973 29454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b2043fb160a4b68b6f201abd87baa27 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:16:38.897078 29454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b2043fb160a4b68b6f201abd87baa27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dda956990bb942c393e67fd846d750b7. Latest consensus state: current_term: 1 leader_uuid: "dda956990bb942c393e67fd846d750b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dda956990bb942c393e67fd846d750b7" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36301 } } peers { permanent_uuid: "6b2043fb160a4b68b6f201abd87baa27" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 38353 } } peers { permanent_uuid: "9908dc7ea9da4e319287bf39f90bb8d2" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 34797 } } }
I20260924 21:16:38.897200 29454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b2043fb160a4b68b6f201abd87baa27 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:16:38.897274 29454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b2043fb160a4b68b6f201abd87baa27 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dda956990bb942c393e67fd846d750b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dda956990bb942c393e67fd846d750b7" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36301 } } peers { permanent_uuid: "6b2043fb160a4b68b6f201abd87baa27" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 38353 } } peers { permanent_uuid: "9908dc7ea9da4e319287bf39f90bb8d2" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 34797 } } }
I20260924 21:16:38.897365 29454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b2043fb160a4b68b6f201abd87baa27 [sys.catalog]: This master's current role is: FOLLOWER
W20260924 21:16:38.898914 29465 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6b2043fb160a4b68b6f201abd87baa27: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 21:16:38.899075 29465 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 21:16:38.902865 29444 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dda956990bb942c393e67fd846d750b7: Generated new TSK 0
I20260924 21:16:38.903054 29444 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:16:38.907477 29467 mvcc.cc:205] Tried to move back new op lower bound from 7333005717035126784 to 7333005716852137984. Current Snapshot: MvccSnapshot[applied={T|T < 7333005716872790016}]
I20260924 21:16:38.907905 29425 sys_catalog.cc:455] T 00000000000000000000000000000000 P dda956990bb942c393e67fd846d750b7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dda956990bb942c393e67fd846d750b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dda956990bb942c393e67fd846d750b7" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 36301 } } peers { permanent_uuid: "6b2043fb160a4b68b6f201abd87baa27" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 38353 } } peers { permanent_uuid: "9908dc7ea9da4e319287bf39f90bb8d2" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 34797 } } }
I20260924 21:16:38.908018 29425 sys_catalog.cc:458] T 00000000000000000000000000000000 P dda956990bb942c393e67fd846d750b7 [sys.catalog]: This master's current role is: LEADER
W20260924 21:16:38.929843 29259 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:16:38.928200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49100 (local address 127.6.210.190:36301)
0924 21:16:38.928245 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:38.928248 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:38.929210 (+ 962us) server_negotiation.cc:407] Connection header received
0924 21:16:38.929309 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:38.929311 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:38.929321 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:38.929350 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:38.929773 (+ 423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49100: BlockingRecv error: recv got EOF from 127.0.0.1:49100 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260924 21:16:38.931058 29334 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:16:38.928603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41084 (local address 127.6.210.189:38353)
0924 21:16:38.928643 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:38.928645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:38.930208 (+ 1563us) server_negotiation.cc:407] Connection header received
0924 21:16:38.930504 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:38.930505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:38.930516 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:38.930544 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:38.930999 (+ 455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41084: BlockingRecv error: recv got EOF from 127.0.0.1:41084 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260924 21:16:38.931628 29409 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:16:38.928931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35532 (local address 127.6.210.188:34797)
0924 21:16:38.928969 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:38.928970 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:38.930358 (+ 1388us) server_negotiation.cc:407] Connection header received
0924 21:16:38.930603 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:38.930604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:38.930614 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:38.930641 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:38.931574 (+ 933us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35532: BlockingRecv error: recv got EOF from 127.0.0.1:35532 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260924 21:16:38.936779 29334 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:16:38.936001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41100 (local address 127.6.210.189:38353)
0924 21:16:38.936039 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:38.936041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:38.936242 (+ 201us) server_negotiation.cc:407] Connection header received
0924 21:16:38.936353 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:38.936354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:38.936373 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:38.936403 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:38.936726 (+ 323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41100: BlockingRecv error: recv got EOF from 127.0.0.1:41100 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260924 21:16:38.939950 29334 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:16:38.939179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41104 (local address 127.6.210.189:38353)
0924 21:16:38.939210 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:38.939212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:38.939463 (+ 251us) server_negotiation.cc:407] Connection header received
0924 21:16:38.939559 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:38.939560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:38.939571 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:38.939599 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:38.939903 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41104: BlockingRecv error: recv got EOF from 127.0.0.1:41104 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260924 21:16:38.943802 29334 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:16:38.943036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41118 (local address 127.6.210.189:38353)
0924 21:16:38.943069 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:38.943071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:38.943311 (+ 240us) server_negotiation.cc:407] Connection header received
0924 21:16:38.943408 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:38.943408 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:38.943418 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:38.943447 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:38.943753 (+ 306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41118: BlockingRecv error: recv got EOF from 127.0.0.1:41118 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260924 21:16:38.949782 29334 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:16:38.949000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41134 (local address 127.6.210.189:38353)
0924 21:16:38.949033 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:38.949035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:38.949271 (+ 236us) server_negotiation.cc:407] Connection header received
0924 21:16:38.949380 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:38.949381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:38.949391 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:38.949421 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:38.949736 (+ 315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41134: BlockingRecv error: recv got EOF from 127.0.0.1:41134 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260924 21:16:38.960268 29334 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:16:38.959373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41136 (local address 127.6.210.189:38353)
0924 21:16:38.959423 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:38.959425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:38.959696 (+ 271us) server_negotiation.cc:407] Connection header received
0924 21:16:38.959797 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:38.959798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:38.959810 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:38.959839 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:38.960204 (+ 365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41136: BlockingRecv error: recv got EOF from 127.0.0.1:41136 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260924 21:16:38.978658 29334 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:16:38.977641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41148 (local address 127.6.210.189:38353)
0924 21:16:38.977684 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:38.977687 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:38.978035 (+ 348us) server_negotiation.cc:407] Connection header received
0924 21:16:38.978146 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:38.978148 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:38.978159 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:38.978189 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:38.978600 (+ 411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41148: BlockingRecv error: recv got EOF from 127.0.0.1:41148 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260924 21:16:39.012751 29334 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:16:39.012015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41154 (local address 127.6.210.189:38353)
0924 21:16:39.012060 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:39.012062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:39.012197 (+ 135us) server_negotiation.cc:407] Connection header received
0924 21:16:39.012272 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:39.012273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:39.012285 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:39.012352 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:39.012677 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41154: BlockingRecv error: recv got EOF from 127.0.0.1:41154 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260924 21:16:39.078821 29334 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:16:39.078080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41158 (local address 127.6.210.189:38353)
0924 21:16:39.078148 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:39.078151 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:39.078223 (+ 72us) server_negotiation.cc:407] Connection header received
0924 21:16:39.078302 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:39.078303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:39.078316 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:39.078405 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:39.078754 (+ 349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41158: BlockingRecv error: recv got EOF from 127.0.0.1:41158 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260924 21:16:39.208885 29334 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:16:39.208063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41166 (local address 127.6.210.189:38353)
0924 21:16:39.208133 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:39.208135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:39.208243 (+ 108us) server_negotiation.cc:407] Connection header received
0924 21:16:39.208277 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:39.208278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:39.208291 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:39.208365 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:39.208825 (+ 460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41166: BlockingRecv error: recv got EOF from 127.0.0.1:41166 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260924 21:16:39.466622 29334 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:16:39.465965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41174 (local address 127.6.210.189:38353)
0924 21:16:39.466041 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:39.466043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:39.466052 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:16:39.466078 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:39.466079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:39.466088 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:39.466154 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:39.466548 (+ 394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41174: BlockingRecv error: recv got EOF from 127.0.0.1:41174 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 21:16:39.818051 29422 catalog_manager.cc:1259] Loaded cluster ID: 0411d07630f343a39ecc6f30f34b218d
I20260924 21:16:39.818133 29422 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9908dc7ea9da4e319287bf39f90bb8d2: loading cluster ID for follower catalog manager: success
I20260924 21:16:39.818985 29422 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9908dc7ea9da4e319287bf39f90bb8d2: acquiring CA information for follower catalog manager: success
I20260924 21:16:39.819331 29422 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9908dc7ea9da4e319287bf39f90bb8d2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 21:16:39.899521 29465 catalog_manager.cc:1259] Loaded cluster ID: 0411d07630f343a39ecc6f30f34b218d
I20260924 21:16:39.899600 29465 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6b2043fb160a4b68b6f201abd87baa27: loading cluster ID for follower catalog manager: success
I20260924 21:16:39.900440 29465 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6b2043fb160a4b68b6f201abd87baa27: acquiring CA information for follower catalog manager: success
I20260924 21:16:39.900753 29465 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6b2043fb160a4b68b6f201abd87baa27: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 21:16:39.982443 29537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:39.979720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41190 (local address 127.6.210.189:38353)
0924 21:16:39.979862 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:39.979864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:39.979901 (+ 37us) server_negotiation.cc:407] Connection header received
0924 21:16:39.979999 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:39.980000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:39.980060 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:39.980146 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:16:39.980319 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:39.980825 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:39.981586 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:39.981802 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:39.981873 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:39.982059 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:16:39.982168 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:16:39.982253 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:39.982323 (+ 70us) server_negotiation.cc:299] Negotiation successful
0924 21:16:39.982344 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":58,"threads_started":1}
I20260924 21:16:39.986205 29543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:39.984118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35548 (local address 127.6.210.188:34797)
0924 21:16:39.984211 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:39.984213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:39.984233 (+ 20us) server_negotiation.cc:407] Connection header received
0924 21:16:39.984266 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:39.984267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:39.984299 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:39.984359 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:16:39.984452 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:39.984905 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:39.985575 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:39.985757 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:39.985804 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:39.985915 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:16:39.986003 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:16:39.986059 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:39.986077 (+ 18us) server_negotiation.cc:299] Negotiation successful
0924 21:16:39.986092 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76,"thread_start_us":42,"threads_started":1}
I20260924 21:16:39.990676 29543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:39.988059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35554 (local address 127.6.210.188:34797)
0924 21:16:39.988243 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:39.988244 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:39.988269 (+ 25us) server_negotiation.cc:407] Connection header received
0924 21:16:39.988291 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:39.988292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:39.988300 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:39.988348 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:16:39.988490 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:39.989160 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:39.989893 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:39.990167 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:39.990229 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:39.990429 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:16:39.990462 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:16:39.990509 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:39.990583 (+ 74us) server_negotiation.cc:299] Negotiation successful
0924 21:16:39.990608 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166}
I20260924 21:16:39.990697 29537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:39.987659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41194 (local address 127.6.210.189:38353)
0924 21:16:39.987694 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:39.987697 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:39.988182 (+ 485us) server_negotiation.cc:407] Connection header received
0924 21:16:39.988208 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:39.988209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:39.988219 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:39.988269 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:16:39.988440 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:39.988893 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:39.989974 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:39.990174 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:39.990226 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:39.990445 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:16:39.990482 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:16:39.990528 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:39.990611 (+ 83us) server_negotiation.cc:299] Negotiation successful
0924 21:16:39.990633 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 21:16:39.991321 29549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:39.987726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49106 (local address 127.6.210.190:36301)
0924 21:16:39.987854 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:39.987856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:39.987865 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:16:39.987908 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:39.987909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:39.987961 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:39.988058 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:16:39.988174 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:39.988991 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:39.989963 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:39.990293 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:39.990358 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:39.990541 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:16:39.991088 (+ 547us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:16:39.991167 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:39.991193 (+ 26us) server_negotiation.cc:299] Negotiation successful
0924 21:16:39.991213 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260924 21:16:39.992702 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 29187
I20260924 21:16:39.998474 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 29262
I20260924 21:16:40.003300 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 29337
2026-09-24T21:16:40Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2837 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:16:40 dist-test-slave-q276 krb5kdc[29554](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:16:40 dist-test-slave-q276 krb5kdc[29554](info): set up 2 sockets
Sep 24 21:16:40 dist-test-slave-q276 krb5kdc[29554](info): commencing operation
krb5kdc: starting...
W20260924 21:16:41.059137 6986 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/krb5kdc/test-user.keytab.
Sep 24 21:16:41 dist-test-slave-q276 krb5kdc[29554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:16:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:16:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.210.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
Entry for principal kudu/127.6.210.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab.
I20260924 21:16:41.180823 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:45659
--webserver_interface=127.6.210.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:46463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:45659,127.6.210.189:39603,127.6.210.188:42225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:41.261181 29567 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 21:16:41.261478 29567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:41.261511 29567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:41.261555 29567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:41.262784 29567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:41.262847 29567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:41.262871 29567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:41.262890 29567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:41.262920 29567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:41.264535 29567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:46463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:45659,127.6.210.189:39603,127.6.210.188:42225
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.190.keytab
--principal=kudu/127.6.210.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.190:45659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.210.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:41.264837 29567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:41.265065 29567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 21:16:41.270067 29567 server_base.cc:1034] running on GCE node
W20260924 21:16:41.270037 29575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:41.270026 29572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:41.270025 29573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:41.270591 29567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:41.270886 29567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:41.272042 29567 hybrid_clock.cc:648] HybridClock initialized: now 1790284601272028 us; error 27 us; skew 500 ppm
Sep 24 21:16:41 dist-test-slave-q276 krb5kdc[29554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284601, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:41.281742 29567 init.cc:377] Logged in from keytab as kudu/127.6.210.190@KRBTEST.COM (short username kudu)
I20260924 21:16:41.282331 29567 webserver.cc:460] Webserver started at http://127.6.210.190:41877/ using document root <none> and password file <none>
I20260924 21:16:41.282579 29567 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:41.282646 29567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:41.282787 29567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:41.283875 29567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-0/data/instance:
uuid: "628f215183a146599ae27580d100f996"
format_stamp: "Formatted at 2026-09-24 21:16:41 on dist-test-slave-q276"
server_key: "717468465deb2ff1f262b6fd6408e238"
server_key_iv: "e3454d83802ef3d59c2229fc9ff80f9a"
server_key_version: "encryptionkey@0"
I20260924 21:16:41.284238 29567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-0/wal/instance:
uuid: "628f215183a146599ae27580d100f996"
format_stamp: "Formatted at 2026-09-24 21:16:41 on dist-test-slave-q276"
server_key: "717468465deb2ff1f262b6fd6408e238"
server_key_iv: "e3454d83802ef3d59c2229fc9ff80f9a"
server_key_version: "encryptionkey@0"
I20260924 21:16:41.285599 29567 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:16:41.286413 29582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:41.286616 29567 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:16:41.286715 29567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-0/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-0/wal
uuid: "628f215183a146599ae27580d100f996"
format_stamp: "Formatted at 2026-09-24 21:16:41 on dist-test-slave-q276"
server_key: "717468465deb2ff1f262b6fd6408e238"
server_key_iv: "e3454d83802ef3d59c2229fc9ff80f9a"
server_key_version: "encryptionkey@0"
I20260924 21:16:41.286798 29567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:41.307457 29567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:41.309706 29567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:41.309864 29567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:41.314143 29567 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.190:45659
I20260924 21:16:41.314200 29634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.190:45659 every 8 connection(s)
I20260924 21:16:41.314577 29567 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-0/data/info.pb
I20260924 21:16:41.315744 29635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:41.315851 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 29567
I20260924 21:16:41.315955 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-0/wal/instance
I20260924 21:16:41.316161 6986 external_mini_cluster.cc:1428] Setting key 5b5e426c77c105dbd8489cd74e22c812
I20260924 21:16:41.317600 29635 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.210.190" port: 45659 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.6.210.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.189@KRBTEST.COM" created.
Sep 24 21:16:41 dist-test-slave-q276 krb5kdc[29554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284601, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:16:41.331465 29638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.318431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:45659 (local address 127.0.0.1:47756)
0924 21:16:41.318597 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:41.318647 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:41.319368 (+ 721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:41.319671 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:41.319675 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:41.320034 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:41.320267 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:41.320273 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.321416 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.321417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:41.322659 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:41.322663 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.323285 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.323286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:41.323376 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:41.324020 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:41.324029 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:41.327536 (+ 3507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:41.329830 (+ 2294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:41.329837 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:41.329839 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:41.330073 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:41.330420 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:41.330421 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:41.330422 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:41.330511 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:41.330997 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:41.331003 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:41.331065 (+ 62us) client_negotiation.cc:769] Sending connection context
0924 21:16:41.331146 (+ 81us) client_negotiation.cc:240] Negotiation successful
0924 21:16:41.331281 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":68,"threads_started":1}
I20260924 21:16:41.331547 29639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.318590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47756 (local address 127.6.210.190:45659)
0924 21:16:41.318714 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:41.318719 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:41.318759 (+ 40us) server_negotiation.cc:407] Connection header received
0924 21:16:41.319533 (+ 774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:41.319535 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:41.319553 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:41.319637 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:41.320393 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.321314 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.322766 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.323165 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.323263 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:41.327632 (+ 4369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:41.327639 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:41.327640 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:41.327646 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:41.329715 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:41.330180 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:41.330183 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:41.330184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:41.330321 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:41.330588 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:41.330589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:41.330589 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:41.330901 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:41.331003 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:41.331225 (+ 222us) server_negotiation.cc:299] Negotiation successful
0924 21:16:41.331374 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":88,"thread_start_us":60,"threads_started":1}
I20260924 21:16:41.332398 29635 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.210.189" port: 39603 } has no permanent_uuid. Determining permanent_uuid...
I20260924 21:16:41.332780 29638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.332651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:39603 (local address 127.0.0.1:60800)
0924 21:16:41.332702 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:41.332752 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.189:39603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260924 21:16:41.333006 29586 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.210.189:39603: connect: Connection refused (error 111)
I20260924 21:16:41.333521 29638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.333408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:39603 (local address 127.0.0.1:60806)
0924 21:16:41.333459 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:41.333491 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.189:39603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260924 21:16:41.333740 29635 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.210.189:39603: Network error: Client connection negotiation failed: client connection to 127.6.210.189:39603: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.189.keytab.
Entry for principal kudu/127.6.210.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.189.keytab.
I20260924 21:16:41.345937 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.189:39603
--webserver_interface=127.6.210.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.189.keytab
--principal=kudu/127.6.210.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:46463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:45659,127.6.210.189:39603,127.6.210.188:42225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:16:41.383014 29635 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.210.189" port: 39603 } attempt: 1
I20260924 21:16:41.383563 29638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.383408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:39603 (local address 127.0.0.1:60810)
0924 21:16:41.383479 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:41.383528 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.189:39603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260924 21:16:41.384001 29638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.383901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:39603 (local address 127.0.0.1:60814)
0924 21:16:41.383947 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:41.383982 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.189:39603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260924 21:16:41.384150 29635 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.210.189:39603: Network error: Client connection negotiation failed: client connection to 127.6.210.189:39603: connect: Connection refused (error 111)
W20260924 21:16:41.427752 29642 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 21:16:41.428062 29642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:41.428104 29642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:41.428146 29642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:41.429345 29642 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:41.429388 29642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:41.429402 29642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:41.429415 29642 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:41.429427 29642 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:41.431067 29642 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:46463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:45659,127.6.210.189:39603,127.6.210.188:42225
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.189.keytab
--principal=kudu/127.6.210.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.189:39603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.210.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:41.431348 29642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:41.431593 29642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:41.436786 29650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:41.436805 29647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:41.436798 29648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:41.437562 29642 server_base.cc:1034] running on GCE node
I20260924 21:16:41.437830 29642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:41.438175 29642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:41.439352 29642 hybrid_clock.cc:648] HybridClock initialized: now 1790284601439336 us; error 49 us; skew 500 ppm
Sep 24 21:16:41 dist-test-slave-q276 krb5kdc[29554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284601, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:41.442126 29642 init.cc:377] Logged in from keytab as kudu/127.6.210.189@KRBTEST.COM (short username kudu)
I20260924 21:16:41.442653 29642 webserver.cc:460] Webserver started at http://127.6.210.189:45653/ using document root <none> and password file <none>
I20260924 21:16:41.442895 29642 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:41.442970 29642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:41.443106 29642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:41.444204 29642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-1/data/instance:
uuid: "da259e86ab4b47f39cabf913e924bfc9"
format_stamp: "Formatted at 2026-09-24 21:16:41 on dist-test-slave-q276"
server_key: "df773da20e32b28f07ad69b65b881b21"
server_key_iv: "90a70029aa3b7581031c602ed776492f"
server_key_version: "encryptionkey@0"
I20260924 21:16:41.444563 29642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-1/wal/instance:
uuid: "da259e86ab4b47f39cabf913e924bfc9"
format_stamp: "Formatted at 2026-09-24 21:16:41 on dist-test-slave-q276"
server_key: "df773da20e32b28f07ad69b65b881b21"
server_key_iv: "90a70029aa3b7581031c602ed776492f"
server_key_version: "encryptionkey@0"
I20260924 21:16:41.445997 29642 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:16:41.446810 29657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:41.447013 29642 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 21:16:41.447109 29642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-1/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-1/wal
uuid: "da259e86ab4b47f39cabf913e924bfc9"
format_stamp: "Formatted at 2026-09-24 21:16:41 on dist-test-slave-q276"
server_key: "df773da20e32b28f07ad69b65b881b21"
server_key_iv: "90a70029aa3b7581031c602ed776492f"
server_key_version: "encryptionkey@0"
I20260924 21:16:41.447191 29642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:41.452348 29635 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.210.189" port: 39603 } attempt: 2
I20260924 21:16:41.452775 29638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.452637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:39603 (local address 127.0.0.1:60826)
0924 21:16:41.452693 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:41.452736 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.189:39603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260924 21:16:41.453094 29638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.453011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:39603 (local address 127.0.0.1:60834)
0924 21:16:41.453049 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:41.453072 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.189:39603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260924 21:16:41.453176 29635 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.210.189:39603: Network error: Client connection negotiation failed: client connection to 127.6.210.189:39603: connect: Connection refused (error 111)
I20260924 21:16:41.461936 29642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:41.464109 29642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:41.464269 29642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:41.468887 29642 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.189:39603
I20260924 21:16:41.468950 29709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.189:39603 every 8 connection(s)
I20260924 21:16:41.469326 29642 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-1/data/info.pb
I20260924 21:16:41.470352 29710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:41.470899 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 29642
I20260924 21:16:41.471009 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-1/wal/instance
I20260924 21:16:41.471197 6986 external_mini_cluster.cc:1428] Setting key f55d1788241898a52d87439c71a2310b
I20260924 21:16:41.472007 29710 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.210.190" port: 45659 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 21:16:41 dist-test-slave-q276 krb5kdc[29554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284601, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.189@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.210.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.210.188@KRBTEST.COM" created.
I20260924 21:16:41.482312 29639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.472740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47764 (local address 127.6.210.190:45659)
0924 21:16:41.472806 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:41.472809 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:41.472986 (+ 177us) server_negotiation.cc:407] Connection header received
0924 21:16:41.473433 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:41.473434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:41.473449 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:41.473525 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:41.474128 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.474976 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.475996 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.476346 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.476427 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:41.479072 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:41.479073 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:41.479073 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:41.479080 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:41.480685 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:41.481133 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:41.481134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:41.481134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:41.481208 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:41.481520 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:41.481521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:41.481521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:41.481671 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:41.481745 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:41.482020 (+ 275us) server_negotiation.cc:299] Negotiation successful
0924 21:16:41.482123 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 21:16:41.482318 29713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.472701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:45659 (local address 127.0.0.1:47764)
0924 21:16:41.472854 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:41.472898 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:41.473345 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:41.473578 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:41.473581 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:41.473841 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:41.474038 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:41.474042 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.475090 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.475091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:41.475912 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:41.475914 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.476441 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.476442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:41.476544 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:41.477125 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:41.477134 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:41.478832 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:41.480773 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:41.480780 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:41.480782 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:41.481013 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:41.481284 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:41.481289 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:41.481289 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:41.481432 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:41.481739 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:41.481746 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:41.481883 (+ 137us) client_negotiation.cc:769] Sending connection context
0924 21:16:41.481977 (+ 94us) client_negotiation.cc:240] Negotiation successful
0924 21:16:41.482106 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":76,"threads_started":1}
I20260924 21:16:41.483143 29710 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.210.189" port: 39603 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 21:16:41 dist-test-slave-q276 krb5kdc[29554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284601, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.189@KRBTEST.COM for kudu/127.6.210.189@KRBTEST.COM
I20260924 21:16:41.491765 29713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.483431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:39603 (local address 127.0.0.1:60850)
0924 21:16:41.483665 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:41.483678 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:41.483740 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:41.483880 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:41.483881 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:41.484050 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:41.484119 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:41.484121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.485402 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.485403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:41.486052 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:41.486054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.486808 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.486809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:41.486876 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:41.487342 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:41.487372 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:41.488601 (+ 1229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:41.490793 (+ 2192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:41.490794 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:41.490794 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:41.490948 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:41.491172 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:41.491173 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:41.491173 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:41.491262 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:41.491533 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:41.491534 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:41.491569 (+ 35us) client_negotiation.cc:769] Sending connection context
0924 21:16:41.491621 (+ 52us) client_negotiation.cc:240] Negotiation successful
0924 21:16:41.491699 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217}
I20260924 21:16:41.491925 29715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.483511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60850 (local address 127.6.210.189:39603)
0924 21:16:41.483670 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:41.483674 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:41.483732 (+ 58us) server_negotiation.cc:407] Connection header received
0924 21:16:41.483803 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:41.483804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:41.483815 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:41.483872 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:41.484199 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.485294 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.486136 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.486721 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.486800 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:41.488686 (+ 1886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:41.488694 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:41.488695 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:41.488703 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:41.490724 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:41.491028 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:41.491031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:41.491031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:41.491087 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:41.491317 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:41.491318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:41.491318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:41.491474 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:41.491542 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:41.491647 (+ 105us) server_negotiation.cc:299] Negotiation successful
0924 21:16:41.491760 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":131,"thread_start_us":92,"threads_started":1}
I20260924 21:16:41.492643 29710 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.210.188" port: 42225 } has no permanent_uuid. Determining permanent_uuid...
I20260924 21:16:41.493006 29713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.492873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:42225 (local address 127.0.0.1:42812)
0924 21:16:41.492941 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:41.492987 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.188:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260924 21:16:41.493136 29659 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.210.188:42225: connect: Connection refused (error 111)
I20260924 21:16:41.493606 29713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.493507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:42225 (local address 127.0.0.1:42818)
0924 21:16:41.493555 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:41.493584 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.188:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260924 21:16:41.493733 29710 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.210.188:42225: Network error: Client connection negotiation failed: client connection to 127.6.210.188:42225: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.210.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.188.keytab.
Entry for principal kudu/127.6.210.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.188.keytab.
I20260924 21:16:41.497691 6986 external_mini_cluster.cc:1326] Running /tmp/dist-test-task5s070Q/build/release/bin/kudu
/tmp/dist-test-task5s070Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.188:42225
--webserver_interface=127.6.210.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.188.keytab
--principal=kudu/127.6.210.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.210.148:46463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.210.190:45659,127.6.210.189:39603,127.6.210.188:42225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:16:41.542960 29710 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.210.188" port: 42225 } attempt: 1
I20260924 21:16:41.543462 29713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.543329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:42225 (local address 127.0.0.1:42832)
0924 21:16:41.543394 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:41.543432 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.188:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260924 21:16:41.543810 29713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.543713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:42225 (local address 127.0.0.1:42840)
0924 21:16:41.543762 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:41.543790 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.188:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260924 21:16:41.543893 29710 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.210.188:42225: Network error: Client connection negotiation failed: client connection to 127.6.210.188:42225: connect: Connection refused (error 111)
I20260924 21:16:41.544298 29635 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.210.189" port: 39603 } attempt: 3
Sep 24 21:16:41 dist-test-slave-q276 krb5kdc[29554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284601, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for kudu/127.6.210.189@KRBTEST.COM
I20260924 21:16:41.551445 29638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.544548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:39603 (local address 127.0.0.1:60864)
0924 21:16:41.544591 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:41.544602 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:41.544667 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:41.544817 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:41.544818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:41.544980 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:41.545044 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:41.545045 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.545733 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.545734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:41.546311 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:41.546314 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.546707 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.546708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:41.546799 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:41.547235 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:41.547241 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:41.548617 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:41.550373 (+ 1756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:41.550374 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:41.550375 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:41.550549 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:41.550765 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:41.550784 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:41.550785 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:41.550848 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:41.551097 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:41.551099 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:41.551146 (+ 47us) client_negotiation.cc:769] Sending connection context
0924 21:16:41.551233 (+ 87us) client_negotiation.cc:240] Negotiation successful
0924 21:16:41.551338 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260924 21:16:41.551491 29715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.544622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60864 (local address 127.6.210.189:39603)
0924 21:16:41.544676 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:41.544678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:41.544691 (+ 13us) server_negotiation.cc:407] Connection header received
0924 21:16:41.544728 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:41.544729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:41.544741 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:41.544787 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:41.545126 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.545650 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.546407 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.546624 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.546705 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:41.548703 (+ 1998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:41.548704 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:41.548705 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:41.548712 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:41.550236 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:41.550632 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:41.550633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:41.550633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:41.550692 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:41.550908 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:41.550909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:41.550910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:41.551028 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:41.551097 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:41.551264 (+ 167us) server_negotiation.cc:299] Negotiation successful
0924 21:16:41.551404 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 21:16:41.552151 29635 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.210.188" port: 42225 } has no permanent_uuid. Determining permanent_uuid...
I20260924 21:16:41.552508 29638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.552403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:42225 (local address 127.0.0.1:42842)
0924 21:16:41.552445 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:41.552481 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.188:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260924 21:16:41.552907 29638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.552767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:42225 (local address 127.0.0.1:42858)
0924 21:16:41.552816 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:41.552843 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.188:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260924 21:16:41.553018 29635 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.210.188:42225: Network error: Client connection negotiation failed: client connection to 127.6.210.188:42225: connect: Connection refused (error 111)
W20260924 21:16:41.580124 29717 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 21:16:41.580333 29717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:41.580350 29717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:41.580369 29717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:41.581521 29717 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:41.581561 29717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:41.581575 29717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:41.581588 29717 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:41.581635 29717 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:41.583132 29717 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.210.148:46463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.210.190:45659,127.6.210.189:39603,127.6.210.188:42225
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/krb5kdc/kudu_127.6.210.188.keytab
--principal=kudu/127.6.210.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.210.188:42225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.210.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:41.583390 29717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:41.583626 29717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 21:16:41.584167 29635 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.210.188" port: 42225 } attempt: 1
I20260924 21:16:41.584602 29638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.584471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:42225 (local address 127.0.0.1:42864)
0924 21:16:41.584533 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:41.584575 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.188:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260924 21:16:41.585008 29638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.584903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:42225 (local address 127.0.0.1:42878)
0924 21:16:41.584950 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:41.584978 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.188:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260924 21:16:41.585147 29635 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.210.188:42225: Network error: Client connection negotiation failed: client connection to 127.6.210.188:42225: connect: Connection refused (error 111)
W20260924 21:16:41.588692 29725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:41.588637 29723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:41.588637 29722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:41.588896 29717 server_base.cc:1034] running on GCE node
I20260924 21:16:41.589144 29717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:41.589488 29717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:41.590626 29717 hybrid_clock.cc:648] HybridClock initialized: now 1790284601590569 us; error 71 us; skew 500 ppm
Sep 24 21:16:41 dist-test-slave-q276 krb5kdc[29554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284601, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:41.593020 29717 init.cc:377] Logged in from keytab as kudu/127.6.210.188@KRBTEST.COM (short username kudu)
I20260924 21:16:41.593487 29717 webserver.cc:460] Webserver started at http://127.6.210.188:42165/ using document root <none> and password file <none>
I20260924 21:16:41.593709 29717 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:41.593773 29717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:41.593914 29717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:41.595023 29717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-2/data/instance:
uuid: "1e416c0c7ae34df19e8ed1db873f641b"
format_stamp: "Formatted at 2026-09-24 21:16:41 on dist-test-slave-q276"
server_key: "a5f01c2943450104d865d6bab5aa3e6c"
server_key_iv: "2a4b1a75d47cec4977d490492ebec9d5"
server_key_version: "encryptionkey@0"
I20260924 21:16:41.595381 29717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-2/wal/instance:
uuid: "1e416c0c7ae34df19e8ed1db873f641b"
format_stamp: "Formatted at 2026-09-24 21:16:41 on dist-test-slave-q276"
server_key: "a5f01c2943450104d865d6bab5aa3e6c"
server_key_iv: "2a4b1a75d47cec4977d490492ebec9d5"
server_key_version: "encryptionkey@0"
I20260924 21:16:41.596689 29717 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:16:41.597427 29732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:41.597589 29717 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:16:41.597685 29717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-2/data,/tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-2/wal
uuid: "1e416c0c7ae34df19e8ed1db873f641b"
format_stamp: "Formatted at 2026-09-24 21:16:41 on dist-test-slave-q276"
server_key: "a5f01c2943450104d865d6bab5aa3e6c"
server_key_iv: "2a4b1a75d47cec4977d490492ebec9d5"
server_key_version: "encryptionkey@0"
I20260924 21:16:41.597754 29717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:41.612033 29710 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.210.188" port: 42225 } attempt: 2
I20260924 21:16:41.612484 29713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.612361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:42225 (local address 127.0.0.1:42892)
0924 21:16:41.612416 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:41.612455 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.188:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260924 21:16:41.612825 29713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.612705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:42225 (local address 127.0.0.1:42896)
0924 21:16:41.612747 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:41.612770 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.210.188:42225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260924 21:16:41.612912 29710 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.210.188:42225: Network error: Client connection negotiation failed: client connection to 127.6.210.188:42225: connect: Connection refused (error 111)
I20260924 21:16:41.614670 29717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:41.616765 29717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:41.616945 29717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:41.621034 29717 rpc_server.cc:307] RPC server started. Bound to: 127.6.210.188:42225
I20260924 21:16:41.621068 29784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.210.188:42225 every 8 connection(s)
I20260924 21:16:41.621464 29717 server_base.cc:1166] Dumped server information to /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-2/data/info.pb
I20260924 21:16:41.622442 6986 external_mini_cluster.cc:1388] Started /tmp/dist-test-task5s070Q/build/release/bin/kudu as pid 29717
I20260924 21:16:41.622558 6986 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task5s070Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284333163253-6986-0/minicluster-data/master-2/wal/instance
I20260924 21:16:41.622588 29785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:41.622767 6986 external_mini_cluster.cc:1428] Setting key 8fda3603696f2b2ef24ffc909f801446
I20260924 21:16:41.625069 29785 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.210.190" port: 45659 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 21:16:41 dist-test-slave-q276 krb5kdc[29554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.188@KRBTEST.COM
Sep 24 21:16:41 dist-test-slave-q276 krb5kdc[29554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284601, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.188@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:16:41.633605 29788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.623276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42904 (local address 127.6.210.188:42225)
0924 21:16:41.623467 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:41.623476 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:41.623500 (+ 24us) server_negotiation.cc:407] Connection header received
0924 21:16:41.624081 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:41.624089 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:41.624137 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:41.624252 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:41.625129 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.626385 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.627251 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.627734 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.627824 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:41.630420 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:41.630428 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:41.630428 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:41.630436 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:41.632025 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:41.632376 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:41.632379 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:41.632380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:41.632462 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:41.632666 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:41.632667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:41.632668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:41.633005 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:41.633142 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:41.633253 (+ 111us) server_negotiation.cc:299] Negotiation successful
0924 21:16:41.633378 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":93,"threads_started":1}
I20260924 21:16:41.633693 29639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.625758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47770 (local address 127.6.210.190:45659)
0924 21:16:41.625812 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:41.625814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:41.626018 (+ 204us) server_negotiation.cc:407] Connection header received
0924 21:16:41.626089 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:41.626090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:41.626101 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:41.626167 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:41.626783 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.627295 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.628233 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.628455 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.628525 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:41.631096 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:41.631097 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:41.631098 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:41.631104 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:41.632494 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:41.632879 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:41.632881 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:41.632881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:41.632937 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:41.633138 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:41.633139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:41.633140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:41.633250 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:41.633320 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:41.633508 (+ 188us) server_negotiation.cc:299] Negotiation successful
0924 21:16:41.633585 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260924 21:16:41.633702 29789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.625721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.190:45659 (local address 127.0.0.1:47770)
0924 21:16:41.625923 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:41.625955 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:41.626035 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:41.626186 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:41.626189 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:41.626580 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:41.626655 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:41.626657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.627398 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.627399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:41.628139 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:41.628142 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.628570 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.628571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:41.628644 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:41.629137 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:41.629145 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:41.631008 (+ 1863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:41.632592 (+ 1584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:41.632600 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:41.632602 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:41.632796 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:41.633012 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:41.633013 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:41.633013 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:41.633078 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:41.633313 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:41.633319 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:41.633369 (+ 50us) client_negotiation.cc:769] Sending connection context
0924 21:16:41.633462 (+ 93us) client_negotiation.cc:240] Negotiation successful
0924 21:16:41.633571 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":142,"threads_started":1}
I20260924 21:16:41.634406 29785 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.210.189" port: 39603 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 21:16:41 dist-test-slave-q276 krb5kdc[29554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284601, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.188@KRBTEST.COM for kudu/127.6.210.189@KRBTEST.COM
Sep 24 21:16:41 dist-test-slave-q276 krb5kdc[29554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.190@KRBTEST.COM
I20260924 21:16:41.642140 29789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.634640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.189:39603 (local address 127.0.0.1:60878)
0924 21:16:41.634680 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:41.634687 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:41.634718 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:41.634859 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:41.634860 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:41.635022 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:41.635066 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:41.635068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.635926 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.635927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:41.636698 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:41.636700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.637171 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.637172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:41.637216 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:41.637610 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:41.637614 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:41.639069 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:41.641099 (+ 2030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:41.641100 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:41.641100 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:41.641281 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:41.641505 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:41.641505 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:41.641506 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:41.641565 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:41.641814 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:41.641815 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:41.641858 (+ 43us) client_negotiation.cc:769] Sending connection context
0924 21:16:41.641940 (+ 82us) client_negotiation.cc:240] Negotiation successful
0924 21:16:41.642042 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260924 21:16:41.642140 29715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.634710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60878 (local address 127.6.210.189:39603)
0924 21:16:41.634754 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:41.634756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:41.634763 (+ 7us) server_negotiation.cc:407] Connection header received
0924 21:16:41.634784 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:41.634785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:41.634793 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:41.634839 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:41.635141 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.635849 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.636796 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.637096 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.637175 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:41.639165 (+ 1990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:41.639167 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:41.639167 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:41.639175 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:41.641011 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:41.641370 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:41.641371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:41.641371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:41.641431 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:41.641632 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:41.641634 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:41.641634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:41.641750 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:41.641822 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:41.641966 (+ 144us) server_negotiation.cc:299] Negotiation successful
0924 21:16:41.642043 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 21:16:41.642764 29785 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.210.188" port: 42225 } has no permanent_uuid. Determining permanent_uuid...
I20260924 21:16:41.642861 29639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.634496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47778 (local address 127.6.210.190:45659)
0924 21:16:41.634560 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:41.634562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:41.634588 (+ 26us) server_negotiation.cc:407] Connection header received
0924 21:16:41.634667 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:41.634668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:41.634679 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:41.634867 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:41.635300 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.635969 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.636903 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.637191 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.637252 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:41.639773 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:41.639774 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:41.639774 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:41.639782 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:41.641367 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:41.641711 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:41.641713 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:41.641713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:41.641775 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:41.641986 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:41.641987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:41.641987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:41.642106 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:41.642323 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:41.642620 (+ 297us) server_negotiation.cc:299] Negotiation successful
0924 21:16:41.642704 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Sep 24 21:16:41 dist-test-slave-q276 krb5kdc[29554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284601, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.188@KRBTEST.COM for kudu/127.6.210.188@KRBTEST.COM
Sep 24 21:16:41 dist-test-slave-q276 krb5kdc[29554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.210.189@KRBTEST.COM
I20260924 21:16:41.650838 29789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.643091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:42225 (local address 127.0.0.1:42906)
0924 21:16:41.643131 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:41.643141 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:41.643184 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:41.643295 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:41.643295 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:41.643455 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:41.643499 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:41.643501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.644525 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.644526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:41.645275 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:41.645277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.645790 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.645791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:41.645830 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:41.646268 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:41.646274 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:41.647945 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:41.649666 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:41.649667 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:41.649668 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:41.649905 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:41.650099 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:41.650100 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:41.650101 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:41.650174 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:41.650545 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:41.650550 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:41.650589 (+ 39us) client_negotiation.cc:769] Sending connection context
0924 21:16:41.650652 (+ 63us) client_negotiation.cc:240] Negotiation successful
0924 21:16:41.650725 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260924 21:16:41.650947 29788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.643140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42906 (local address 127.6.210.188:42225)
0924 21:16:41.643184 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:41.643186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:41.643196 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:16:41.643224 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:41.643225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:41.643236 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:41.643293 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:41.643570 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.644439 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.645346 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.645714 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.645806 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:41.648034 (+ 2228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:41.648036 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:41.648036 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:41.648043 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:41.649570 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:41.649967 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:41.649968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:41.649969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:41.650029 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:41.650258 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:41.650259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:41.650259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:41.650434 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:41.650520 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:41.650714 (+ 194us) server_negotiation.cc:299] Negotiation successful
0924 21:16:41.650801 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 21:16:41.652230 29715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.643787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60880 (local address 127.6.210.189:39603)
0924 21:16:41.643820 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:41.643821 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:41.643828 (+ 7us) server_negotiation.cc:407] Connection header received
0924 21:16:41.643886 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:41.643887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:41.643894 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:41.643934 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:41.644514 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.645231 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.646317 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.646629 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.646715 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:41.649201 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:41.649202 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:41.649202 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:41.649209 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:41.651039 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:41.651443 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:41.651444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:41.651445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:41.651510 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:41.651691 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:41.651692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:41.651692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:41.651829 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:41.652012 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:41.652051 (+ 39us) server_negotiation.cc:299] Negotiation successful
0924 21:16:41.652128 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 21:16:41.652693 29785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e416c0c7ae34df19e8ed1db873f641b: Bootstrap starting.
I20260924 21:16:41.653486 29785 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e416c0c7ae34df19e8ed1db873f641b: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:41.653728 29785 log.cc:824] T 00000000000000000000000000000000 P 1e416c0c7ae34df19e8ed1db873f641b: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:41.654752 29785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e416c0c7ae34df19e8ed1db873f641b: No bootstrap required, opened a new log
I20260924 21:16:41.655488 6986 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:16:41.656985 29785 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e416c0c7ae34df19e8ed1db873f641b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "628f215183a146599ae27580d100f996" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 45659 } } peers { permanent_uuid: "da259e86ab4b47f39cabf913e924bfc9" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 39603 } } peers { permanent_uuid: "1e416c0c7ae34df19e8ed1db873f641b" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 42225 } }
I20260924 21:16:41.657159 29785 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e416c0c7ae34df19e8ed1db873f641b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e416c0c7ae34df19e8ed1db873f641b, State: Initialized, Role: FOLLOWER
I20260924 21:16:41.657251 29785 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e416c0c7ae34df19e8ed1db873f641b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "628f215183a146599ae27580d100f996" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 45659 } } peers { permanent_uuid: "da259e86ab4b47f39cabf913e924bfc9" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 39603 } } peers { permanent_uuid: "1e416c0c7ae34df19e8ed1db873f641b" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 42225 } }
I20260924 21:16:41.657546 29791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e416c0c7ae34df19e8ed1db873f641b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "628f215183a146599ae27580d100f996" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 45659 } } peers { permanent_uuid: "da259e86ab4b47f39cabf913e924bfc9" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 39603 } } peers { permanent_uuid: "1e416c0c7ae34df19e8ed1db873f641b" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 42225 } } }
I20260924 21:16:41.657612 29785 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e416c0c7ae34df19e8ed1db873f641b [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:41.657670 29791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e416c0c7ae34df19e8ed1db873f641b [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:16:41.656752 29609 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "628f215183a146599ae27580d100f996"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47778
W20260924 21:16:41.659642 29802 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1e416c0c7ae34df19e8ed1db873f641b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 21:16:41.659901 29802 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 21:16:41.660296 29635 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.210.188" port: 42225 } attempt: 2
I20260924 21:16:41.660086 29717 master_runner.cc:186] Error getting master registration for 127.6.210.190:45659: OK, instance_id { permanent_uuid: "628f215183a146599ae27580d100f996" instance_seqno: 1790284601312979 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 21:16:41.663249 29717 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Sep 24 21:16:41 dist-test-slave-q276 krb5kdc[29554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284601, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.190@KRBTEST.COM for kudu/127.6.210.188@KRBTEST.COM
I20260924 21:16:41.668476 29788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.660646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42922 (local address 127.6.210.188:42225)
0924 21:16:41.660683 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:41.660685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:41.660695 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:16:41.660717 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:41.660718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:41.660729 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:41.660790 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:41.661074 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.661780 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.662823 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.663011 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.663100 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:41.665325 (+ 2225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:41.665327 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:41.665327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:41.665334 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:41.667125 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:41.667570 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:41.667571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:41.667572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:41.667634 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:41.667876 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:41.667877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:41.667877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:41.668048 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:41.668119 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:41.668294 (+ 175us) server_negotiation.cc:299] Negotiation successful
0924 21:16:41.668375 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 21:16:41.668476 29638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.660550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:42225 (local address 127.0.0.1:42922)
0924 21:16:41.660596 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:41.660608 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:41.660651 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:41.660793 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:41.660793 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:41.660954 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:41.660993 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:41.660995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.661880 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.661881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:41.662727 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:41.662729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.663099 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.663100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:41.663158 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:41.663589 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:41.663595 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:41.665220 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:41.667216 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:41.667217 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:41.667217 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:41.667480 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:41.667708 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:41.667709 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:41.667709 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:41.667788 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:41.668121 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:41.668122 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:41.668176 (+ 54us) client_negotiation.cc:769] Sending connection context
0924 21:16:41.668262 (+ 86us) client_negotiation.cc:240] Negotiation successful
0924 21:16:41.668369 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260924 21:16:41.670423 29635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 628f215183a146599ae27580d100f996: Bootstrap starting.
I20260924 21:16:41.671314 29635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 628f215183a146599ae27580d100f996: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:41.671665 29635 log.cc:824] T 00000000000000000000000000000000 P 628f215183a146599ae27580d100f996: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:41.672588 29635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 628f215183a146599ae27580d100f996: No bootstrap required, opened a new log
I20260924 21:16:41.674088 29635 raft_consensus.cc:348] T 00000000000000000000000000000000 P 628f215183a146599ae27580d100f996 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "628f215183a146599ae27580d100f996" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 45659 } } peers { permanent_uuid: "da259e86ab4b47f39cabf913e924bfc9" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 39603 } } peers { permanent_uuid: "1e416c0c7ae34df19e8ed1db873f641b" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 42225 } }
I20260924 21:16:41.674245 29635 raft_consensus.cc:729] T 00000000000000000000000000000000 P 628f215183a146599ae27580d100f996 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 628f215183a146599ae27580d100f996, State: Initialized, Role: FOLLOWER
I20260924 21:16:41.674369 29635 consensus_queue.cc:260] T 00000000000000000000000000000000 P 628f215183a146599ae27580d100f996 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "628f215183a146599ae27580d100f996" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 45659 } } peers { permanent_uuid: "da259e86ab4b47f39cabf913e924bfc9" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 39603 } } peers { permanent_uuid: "1e416c0c7ae34df19e8ed1db873f641b" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 42225 } }
I20260924 21:16:41.674710 29635 sys_catalog.cc:564] T 00000000000000000000000000000000 P 628f215183a146599ae27580d100f996 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:41.674675 29805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 628f215183a146599ae27580d100f996 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "628f215183a146599ae27580d100f996" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 45659 } } peers { permanent_uuid: "da259e86ab4b47f39cabf913e924bfc9" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 39603 } } peers { permanent_uuid: "1e416c0c7ae34df19e8ed1db873f641b" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 42225 } } }
I20260924 21:16:41.674750 29609 raft_consensus.cc:482] T 00000000000000000000000000000000 P 628f215183a146599ae27580d100f996 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 21:16:41.674850 29805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 628f215183a146599ae27580d100f996 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:16:41.674906 29609 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 628f215183a146599ae27580d100f996 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:41.675768 29609 raft_consensus.cc:504] T 00000000000000000000000000000000 P 628f215183a146599ae27580d100f996 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "628f215183a146599ae27580d100f996" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 45659 } } peers { permanent_uuid: "da259e86ab4b47f39cabf913e924bfc9" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 39603 } } peers { permanent_uuid: "1e416c0c7ae34df19e8ed1db873f641b" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 42225 } }
I20260924 21:16:41.676326 29759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "628f215183a146599ae27580d100f996" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1e416c0c7ae34df19e8ed1db873f641b"
I20260924 21:16:41.676489 29759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e416c0c7ae34df19e8ed1db873f641b [term 0 FOLLOWER]: Advancing to term 1
W20260924 21:16:41.677062 29816 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 628f215183a146599ae27580d100f996: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 21:16:41.677120 29816 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 21:16:41.677541 29759 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1e416c0c7ae34df19e8ed1db873f641b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 628f215183a146599ae27580d100f996 in term 1.
I20260924 21:16:41.677870 29584 leader_election.cc:302] T 00000000000000000000000000000000 P 628f215183a146599ae27580d100f996 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e416c0c7ae34df19e8ed1db873f641b, 628f215183a146599ae27580d100f996; no voters:
I20260924 21:16:41.678018 29805 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 628f215183a146599ae27580d100f996 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:41.678188 29805 raft_consensus.cc:686] T 00000000000000000000000000000000 P 628f215183a146599ae27580d100f996 [term 1 LEADER]: Becoming Leader. State: Replica: 628f215183a146599ae27580d100f996, State: Running, Role: LEADER
I20260924 21:16:41.678304 29805 consensus_queue.cc:237] T 00000000000000000000000000000000 P 628f215183a146599ae27580d100f996 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "628f215183a146599ae27580d100f996" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 45659 } } peers { permanent_uuid: "da259e86ab4b47f39cabf913e924bfc9" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 39603 } } peers { permanent_uuid: "1e416c0c7ae34df19e8ed1db873f641b" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 42225 } }
I20260924 21:16:41.678596 29609 leader_election.cc:288] T 00000000000000000000000000000000 P 628f215183a146599ae27580d100f996 [CANDIDATE]: Term 1 election: Requested vote from peers da259e86ab4b47f39cabf913e924bfc9 (127.6.210.189:39603), 1e416c0c7ae34df19e8ed1db873f641b (127.6.210.188:42225)
I20260924 21:16:41.678907 29805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 628f215183a146599ae27580d100f996 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 628f215183a146599ae27580d100f996. Latest consensus state: current_term: 1 leader_uuid: "628f215183a146599ae27580d100f996" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "628f215183a146599ae27580d100f996" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 45659 } } peers { permanent_uuid: "da259e86ab4b47f39cabf913e924bfc9" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 39603 } } peers { permanent_uuid: "1e416c0c7ae34df19e8ed1db873f641b" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 42225 } } }
I20260924 21:16:41.678993 29805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 628f215183a146599ae27580d100f996 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:41.680169 29820 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:16:41.680408 29567 master_runner.cc:186] Error getting master registration for 127.6.210.189:39603: OK, instance_id { permanent_uuid: "da259e86ab4b47f39cabf913e924bfc9" instance_seqno: 1790284601467707 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 21:16:41.682628 29567 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 21:16:41.682857 29820 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:16:41.684966 29759 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1e416c0c7ae34df19e8ed1db873f641b [term 1 FOLLOWER]: Refusing update from remote peer 628f215183a146599ae27580d100f996: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:16:41.685400 29805 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 628f215183a146599ae27580d100f996 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e416c0c7ae34df19e8ed1db873f641b" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:16:41.686924 29791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e416c0c7ae34df19e8ed1db873f641b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 628f215183a146599ae27580d100f996. Latest consensus state: current_term: 1 leader_uuid: "628f215183a146599ae27580d100f996" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "628f215183a146599ae27580d100f996" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 45659 } } peers { permanent_uuid: "da259e86ab4b47f39cabf913e924bfc9" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 39603 } } peers { permanent_uuid: "1e416c0c7ae34df19e8ed1db873f641b" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 42225 } } }
I20260924 21:16:41.687031 29791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e416c0c7ae34df19e8ed1db873f641b [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:16:41.699697 29639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.682445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47782 (local address 127.6.210.190:45659)
0924 21:16:41.682491 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:41.682493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:41.685436 (+ 2943us) server_negotiation.cc:407] Connection header received
0924 21:16:41.685547 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:41.685548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:41.685559 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:41.685594 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:41.688412 (+ 2818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.689125 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.690294 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.690601 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.690653 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:41.695719 (+ 5066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:41.695722 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:41.695722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:41.695730 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:41.697564 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:41.698651 (+ 1087us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:41.698652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:41.698652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:41.698718 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:41.699010 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:41.699011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:41.699011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:41.699157 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:41.699252 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:41.699473 (+ 221us) server_negotiation.cc:299] Negotiation successful
0924 21:16:41.699582 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 21:16:41.700285 29818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 628f215183a146599ae27580d100f996 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "628f215183a146599ae27580d100f996" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "628f215183a146599ae27580d100f996" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 45659 } } peers { permanent_uuid: "da259e86ab4b47f39cabf913e924bfc9" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 39603 } } peers { permanent_uuid: "1e416c0c7ae34df19e8ed1db873f641b" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 42225 } } }
I20260924 21:16:41.700431 29818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 628f215183a146599ae27580d100f996 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:41.700886 29791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e416c0c7ae34df19e8ed1db873f641b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "628f215183a146599ae27580d100f996" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "628f215183a146599ae27580d100f996" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 45659 } } peers { permanent_uuid: "da259e86ab4b47f39cabf913e924bfc9" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 39603 } } peers { permanent_uuid: "1e416c0c7ae34df19e8ed1db873f641b" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 42225 } } }
I20260924 21:16:41.701009 29791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e416c0c7ae34df19e8ed1db873f641b [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:16:41.702198 29715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.679951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60892 (local address 127.6.210.189:39603)
0924 21:16:41.680076 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:41.680079 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:41.684251 (+ 4172us) server_negotiation.cc:407] Connection header received
0924 21:16:41.684385 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:41.684386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:41.684400 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:41.684438 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:41.686470 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.687180 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.688262 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.688568 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.688638 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:41.692778 (+ 4140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:41.692780 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:41.692780 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:41.692803 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:41.694672 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:41.698647 (+ 3975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:41.698648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:41.698649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:41.698726 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:41.699070 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:41.699071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:41.699071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:41.699224 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:41.699318 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:41.701984 (+ 2666us) server_negotiation.cc:299] Negotiation successful
0924 21:16:41.702080 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260924 21:16:41.701771 29828 mvcc.cc:205] Tried to move back new op lower bound from 7333005728497516544 to 7333005728475660288. Current Snapshot: MvccSnapshot[applied={T|T < 7333005728497516544}]
I20260924 21:16:41.704025 29710 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.210.188" port: 42225 } attempt: 3
I20260924 21:16:41.704149 29820 catalog_manager.cc:1347] Generated new cluster ID: 306fc1a784704489a44e113d54c1b2c5
I20260924 21:16:41.704212 29820 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 21:16:41 dist-test-slave-q276 krb5kdc[29554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284601, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.210.189@KRBTEST.COM for kudu/127.6.210.188@KRBTEST.COM
I20260924 21:16:41.714159 29788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.682730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42930 (local address 127.6.210.188:42225)
0924 21:16:41.682778 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:41.682780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:41.688358 (+ 5578us) server_negotiation.cc:407] Connection header received
0924 21:16:41.689673 (+ 1315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:41.689675 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:41.689687 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:41.689743 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:41.691079 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.691772 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.695742 (+ 3970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.696314 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.696369 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:41.700721 (+ 4352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:41.700723 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:41.700723 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:41.700742 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:41.702671 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:41.703513 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:41.703514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:41.703515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:41.703579 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:41.703849 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:41.703850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:41.703850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:41.703990 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:41.704077 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:41.713948 (+ 9871us) server_negotiation.cc:299] Negotiation successful
0924 21:16:41.714043 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 21:16:41.714367 29713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.704313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.210.188:42225 (local address 127.0.0.1:42942)
0924 21:16:41.704362 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:41.704373 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:41.704418 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:41.704714 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:41.704715 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:41.704889 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:41.704939 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:41.704941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.706426 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.706427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:41.707210 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:41.707212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.707936 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.707937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:41.708006 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:41.708460 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:41.708464 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:41.709941 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:41.712896 (+ 2955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:41.712897 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:41.712897 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:41.713115 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:41.713485 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:41.713486 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:41.713486 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:41.713563 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:41.714040 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:41.714041 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:41.714085 (+ 44us) client_negotiation.cc:769] Sending connection context
0924 21:16:41.714137 (+ 52us) client_negotiation.cc:240] Negotiation successful
0924 21:16:41.714246 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260924 21:16:41.715167 29833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:41.704359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42942 (local address 127.6.210.188:42225)
0924 21:16:41.704502 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:41.704504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:41.704535 (+ 31us) server_negotiation.cc:407] Connection header received
0924 21:16:41.704556 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:41.704556 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:41.704567 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:41.705018 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:41.705075 (+ 57us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.706304 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.707330 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:41.707829 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:41.708182 (+ 353us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:41.710057 (+ 1875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:41.710059 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:41.710059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:41.710066 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:41.712759 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:41.713251 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:41.713253 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:41.713253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:41.713358 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:41.713659 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:41.713660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:41.713660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:41.713916 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:41.714625 (+ 709us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:41.714702 (+ 77us) server_negotiation.cc:299] Negotiation successful
0924 21:16:41.714922 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
I20260924 21:16:41.716917 29710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da259e86ab4b47f39cabf913e924bfc9: Bootstrap starting.
I20260924 21:16:41.717732 29710 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da259e86ab4b47f39cabf913e924bfc9: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:41.718070 29710 log.cc:824] T 00000000000000000000000000000000 P da259e86ab4b47f39cabf913e924bfc9: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:41.718766 29820 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:16:41.719583 29820 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:16:41.723733 29710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da259e86ab4b47f39cabf913e924bfc9: No bootstrap required, opened a new log
I20260924 21:16:41.724537 29820 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 628f215183a146599ae27580d100f996: Generated new TSK 0
I20260924 21:16:41.724965 29820 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:16:41.726110 29710 raft_consensus.cc:348] T 00000000000000000000000000000000 P da259e86ab4b47f39cabf913e924bfc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "628f215183a146599ae27580d100f996" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 45659 } } peers { permanent_uuid: "da259e86ab4b47f39cabf913e924bfc9" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 39603 } } peers { permanent_uuid: "1e416c0c7ae34df19e8ed1db873f641b" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 42225 } }
I20260924 21:16:41.726318 29710 raft_consensus.cc:729] T 00000000000000000000000000000000 P da259e86ab4b47f39cabf913e924bfc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da259e86ab4b47f39cabf913e924bfc9, State: Initialized, Role: FOLLOWER
I20260924 21:16:41.726424 29710 consensus_queue.cc:260] T 00000000000000000000000000000000 P da259e86ab4b47f39cabf913e924bfc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "628f215183a146599ae27580d100f996" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 45659 } } peers { permanent_uuid: "da259e86ab4b47f39cabf913e924bfc9" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 39603 } } peers { permanent_uuid: "1e416c0c7ae34df19e8ed1db873f641b" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 42225 } }
I20260924 21:16:41.726786 29710 sys_catalog.cc:564] T 00000000000000000000000000000000 P da259e86ab4b47f39cabf913e924bfc9 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:41.728785 29684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "628f215183a146599ae27580d100f996" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "da259e86ab4b47f39cabf913e924bfc9"
I20260924 21:16:41.728681 29683 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da259e86ab4b47f39cabf913e924bfc9 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:41.730146 29683 raft_consensus.cc:1261] T 00000000000000000000000000000000 P da259e86ab4b47f39cabf913e924bfc9 [term 1 FOLLOWER]: Refusing update from remote peer 628f215183a146599ae27580d100f996: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:16:41.730508 29805 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 628f215183a146599ae27580d100f996 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da259e86ab4b47f39cabf913e924bfc9" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 39603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:16:41.731101 29684 raft_consensus.cc:2454] T 00000000000000000000000000000000 P da259e86ab4b47f39cabf913e924bfc9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 628f215183a146599ae27580d100f996 in term 1.
I20260924 21:16:41.734350 29835 sys_catalog.cc:455] T 00000000000000000000000000000000 P da259e86ab4b47f39cabf913e924bfc9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 628f215183a146599ae27580d100f996. Latest consensus state: current_term: 1 leader_uuid: "628f215183a146599ae27580d100f996" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "628f215183a146599ae27580d100f996" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 45659 } } peers { permanent_uuid: "da259e86ab4b47f39cabf913e924bfc9" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 39603 } } peers { permanent_uuid: "1e416c0c7ae34df19e8ed1db873f641b" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 42225 } } }
I20260924 21:16:41.734457 29835 sys_catalog.cc:458] T 00000000000000000000000000000000 P da259e86ab4b47f39cabf913e924bfc9 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:16:41.734508 29835 sys_catalog.cc:455] T 00000000000000000000000000000000 P da259e86ab4b47f39cabf913e924bfc9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "628f215183a146599ae27580d100f996" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "628f215183a146599ae27580d100f996" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 45659 } } peers { permanent_uuid: "da259e86ab4b47f39cabf913e924bfc9" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 39603 } } peers { permanent_uuid: "1e416c0c7ae34df19e8ed1db873f641b" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 42225 } } }
I20260924 21:16:41.734563 29835 sys_catalog.cc:458] T 00000000000000000000000000000000 P da259e86ab4b47f39cabf913e924bfc9 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:16:41.735122 29847 sys_catalog.cc:455] T 00000000000000000000000000000000 P da259e86ab4b47f39cabf913e924bfc9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "628f215183a146599ae27580d100f996" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "628f215183a146599ae27580d100f996" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 45659 } } peers { permanent_uuid: "da259e86ab4b47f39cabf913e924bfc9" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 39603 } } peers { permanent_uuid: "1e416c0c7ae34df19e8ed1db873f641b" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 42225 } } }
I20260924 21:16:41.735234 29847 sys_catalog.cc:458] T 00000000000000000000000000000000 P da259e86ab4b47f39cabf913e924bfc9 [sys.catalog]: This master's current role is: FOLLOWER
W20260924 21:16:41.735329 29846 catalog_manager.cc:1558] T 00000000000000000000000000000000 P da259e86ab4b47f39cabf913e924bfc9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 21:16:41.735394 29846 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260924 21:16:41.750540 29715 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:16:41.748631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60908 (local address 127.6.210.189:39603)
0924 21:16:41.748683 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:41.748685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:41.749786 (+ 1101us) server_negotiation.cc:407] Connection header received
0924 21:16:41.749910 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:41.749911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:41.749924 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:41.749959 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:41.750469 (+ 510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60908: BlockingRecv error: recv got EOF from 127.0.0.1:60908 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260924 21:16:41.751379 29639 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:16:41.749063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47792 (local address 127.6.210.190:45659)
0924 21:16:41.749108 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:41.749110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:41.750816 (+ 1706us) server_negotiation.cc:407] Connection header received
0924 21:16:41.750926 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:41.750928 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:41.750939 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:41.750973 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:41.751313 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47792: BlockingRecv error: recv got EOF from 127.0.0.1:47792 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260924 21:16:41.752295 29833 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:16:41.749461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42952 (local address 127.6.210.188:42225)
0924 21:16:41.749506 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:41.749507 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:41.751606 (+ 2099us) server_negotiation.cc:407] Connection header received
0924 21:16:41.751777 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:41.751778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:41.751793 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:41.751827 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:41.752221 (+ 394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42952: BlockingRecv error: recv got EOF from 127.0.0.1:42952 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260924 21:16:41.754801 29715 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:16:41.753921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60910 (local address 127.6.210.189:39603)
0924 21:16:41.753958 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:41.753959 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:41.754221 (+ 262us) server_negotiation.cc:407] Connection header received
0924 21:16:41.754376 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:41.754377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:41.754396 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:41.754429 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:41.754752 (+ 323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60910: BlockingRecv error: recv got EOF from 127.0.0.1:60910 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260924 21:16:41.757922 29715 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:16:41.756994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60920 (local address 127.6.210.189:39603)
0924 21:16:41.757028 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:41.757029 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:41.757322 (+ 293us) server_negotiation.cc:407] Connection header received
0924 21:16:41.757481 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:41.757482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:41.757493 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:41.757526 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:41.757842 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60920: BlockingRecv error: recv got EOF from 127.0.0.1:60920 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260924 21:16:41.761793 29715 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:16:41.761010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60924 (local address 127.6.210.189:39603)
0924 21:16:41.761044 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:41.761046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:41.761301 (+ 255us) server_negotiation.cc:407] Connection header received
0924 21:16:41.761404 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:41.761405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:41.761415 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:41.761447 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:41.761751 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60924: BlockingRecv error: recv got EOF from 127.0.0.1:60924 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260924 21:16:41.767752 29715 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:16:41.766900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60932 (local address 127.6.210.189:39603)
0924 21:16:41.766938 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:41.766940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:41.767185 (+ 245us) server_negotiation.cc:407] Connection header received
0924 21:16:41.767311 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:41.767312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:41.767322 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:41.767356 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:41.767703 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60932: BlockingRecv error: recv got EOF from 127.0.0.1:60932 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260924 21:16:41.777956 29715 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:16:41.777016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60942 (local address 127.6.210.189:39603)
0924 21:16:41.777058 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:41.777060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:41.777377 (+ 317us) server_negotiation.cc:407] Connection header received
0924 21:16:41.777488 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:41.777489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:41.777501 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:41.777535 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:41.777901 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60942: BlockingRecv error: recv got EOF from 127.0.0.1:60942 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260924 21:16:41.796373 29715 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:16:41.795359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60948 (local address 127.6.210.189:39603)
0924 21:16:41.795405 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:41.795407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:41.795723 (+ 316us) server_negotiation.cc:407] Connection header received
0924 21:16:41.795845 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:41.795846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:41.795860 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:41.795895 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:41.796311 (+ 416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60948: BlockingRecv error: recv got EOF from 127.0.0.1:60948 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260924 21:16:41.830945 29715 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:16:41.829842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60952 (local address 127.6.210.189:39603)
0924 21:16:41.829895 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:41.829898 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:41.830215 (+ 317us) server_negotiation.cc:407] Connection header received
0924 21:16:41.830375 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:41.830377 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:41.830391 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:41.830427 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:41.830881 (+ 454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60952: BlockingRecv error: recv got EOF from 127.0.0.1:60952 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260924 21:16:41.897333 29715 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:16:41.896562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60956 (local address 127.6.210.189:39603)
0924 21:16:41.896627 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:41.896629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:41.896793 (+ 164us) server_negotiation.cc:407] Connection header received
0924 21:16:41.896841 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:41.896842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:41.896856 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:41.896935 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:41.897284 (+ 349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60956: BlockingRecv error: recv got EOF from 127.0.0.1:60956 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260924 21:16:42.027451 29715 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:16:42.026586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60966 (local address 127.6.210.189:39603)
0924 21:16:42.026671 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:42.026674 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:42.026818 (+ 144us) server_negotiation.cc:407] Connection header received
0924 21:16:42.026849 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:42.026850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:42.026861 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:42.026920 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:42.027369 (+ 449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60966: BlockingRecv error: recv got EOF from 127.0.0.1:60966 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 21:16:42.162894 29805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 628f215183a146599ae27580d100f996 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "628f215183a146599ae27580d100f996" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "628f215183a146599ae27580d100f996" member_type: VOTER last_known_addr { host: "127.6.210.190" port: 45659 } } peers { permanent_uuid: "da259e86ab4b47f39cabf913e924bfc9" member_type: VOTER last_known_addr { host: "127.6.210.189" port: 39603 } } peers { permanent_uuid: "1e416c0c7ae34df19e8ed1db873f641b" member_type: VOTER last_known_addr { host: "127.6.210.188" port: 42225 } } }
I20260924 21:16:42.163066 29805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 628f215183a146599ae27580d100f996 [sys.catalog]: This master's current role is: LEADER
W20260924 21:16:42.285123 29715 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:16:42.284475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60974 (local address 127.6.210.189:39603)
0924 21:16:42.284536 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:42.284538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:42.284581 (+ 43us) server_negotiation.cc:407] Connection header received
0924 21:16:42.284653 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:42.284654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:42.284664 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:42.284716 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:42.285055 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60974: BlockingRecv error: recv got EOF from 127.0.0.1:60974 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 21:16:42.660333 29802 catalog_manager.cc:1259] Loaded cluster ID: 306fc1a784704489a44e113d54c1b2c5
I20260924 21:16:42.660408 29802 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1e416c0c7ae34df19e8ed1db873f641b: loading cluster ID for follower catalog manager: success
I20260924 21:16:42.661178 29802 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1e416c0c7ae34df19e8ed1db873f641b: acquiring CA information for follower catalog manager: success
I20260924 21:16:42.661432 29802 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1e416c0c7ae34df19e8ed1db873f641b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 21:16:42.735816 29846 catalog_manager.cc:1259] Loaded cluster ID: 306fc1a784704489a44e113d54c1b2c5
I20260924 21:16:42.735904 29846 catalog_manager.cc:1552] T 00000000000000000000000000000000 P da259e86ab4b47f39cabf913e924bfc9: loading cluster ID for follower catalog manager: success
I20260924 21:16:42.736769 29846 catalog_manager.cc:1574] T 00000000000000000000000000000000 P da259e86ab4b47f39cabf913e924bfc9: acquiring CA information for follower catalog manager: success
I20260924 21:16:42.737082 29846 catalog_manager.cc:1602] T 00000000000000000000000000000000 P da259e86ab4b47f39cabf913e924bfc9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 21:16:42.800774 29920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:42.798174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60988 (local address 127.6.210.189:39603)
0924 21:16:42.798363 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:42.798365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:42.798378 (+ 13us) server_negotiation.cc:407] Connection header received
0924 21:16:42.798407 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:42.798407 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:42.798465 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:42.798553 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:16:42.798698 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:42.799260 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:42.799987 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:42.800194 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:42.800262 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:42.800436 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:16:42.800545 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:16:42.800610 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:42.800665 (+ 55us) server_negotiation.cc:299] Negotiation successful
0924 21:16:42.800690 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":70,"threads_started":1}
I20260924 21:16:42.805047 29926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:42.802621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42968 (local address 127.6.210.188:42225)
0924 21:16:42.802742 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:42.802744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:42.802753 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:16:42.802800 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:42.802801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:42.802855 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:42.802936 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:16:42.803036 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:42.803515 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:42.804239 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:42.804458 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:42.804540 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:42.804706 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:16:42.804821 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:16:42.804905 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:42.804926 (+ 21us) server_negotiation.cc:299] Negotiation successful
0924 21:16:42.804949 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":58,"threads_started":1}
I20260924 21:16:42.809476 29926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:42.806764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42982 (local address 127.6.210.188:42225)
0924 21:16:42.806799 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:42.806801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:42.806958 (+ 157us) server_negotiation.cc:407] Connection header received
0924 21:16:42.807053 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:42.807054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:42.807064 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:42.807111 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:16:42.807266 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:42.807741 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:42.808720 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:42.809007 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:42.809084 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:42.809195 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:16:42.809227 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:16:42.809280 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:42.809365 (+ 85us) server_negotiation.cc:299] Negotiation successful
0924 21:16:42.809393 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 21:16:42.809695 29920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:42.806664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32772 (local address 127.6.210.189:39603)
0924 21:16:42.806722 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:42.806725 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:42.807011 (+ 286us) server_negotiation.cc:407] Connection header received
0924 21:16:42.807091 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:42.807092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:42.807103 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:42.807154 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:16:42.807313 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:42.808036 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:42.808916 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:42.809218 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:42.809276 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:42.809412 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:16:42.809445 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:16:42.809472 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:42.809591 (+ 119us) server_negotiation.cc:299] Negotiation successful
0924 21:16:42.809618 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260924 21:16:42.810144 29932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:42.806619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47794 (local address 127.6.210.190:45659)
0924 21:16:42.806849 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:42.806851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:42.806859 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:16:42.806905 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:42.806906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:42.806961 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:42.807073 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:16:42.807238 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:42.808040 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:42.809056 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:42.809383 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:42.809646 (+ 263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:42.809725 (+ 79us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:16:42.809857 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:16:42.809953 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:42.810022 (+ 69us) server_negotiation.cc:299] Negotiation successful
0924 21:16:42.810068 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":66,"threads_started":1}
I20260924 21:16:42.811956 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 29567
I20260924 21:16:42.817178 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 29642
I20260924 21:16:42.821820 6986 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task5s070Q/build/release/bin/kudu with pid 29717
2026-09-24T21:16:42Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2819 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5657 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (269667 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST